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Preface 

I feel  that  the  Air  Force  faces  a great  challenge  in  developing 
advanced  technology  weapons  within  the  environment  established  by  today's 
national  priorities.  As  the  resources  for  military  research  and  develop- 
ment decline,  wo  must  make  every  effort  to  insure  that  maximum  use  is 
made  of  the  technological  advances  achieved  by  the  research  and  develop- 
ment community  of  this  country.  The  assessment  of  advanced  development 
technology  programs  is  an  action  taken  in  an  attempt  to  enhance  the  use 
of  these  technological  advances.  The  effectiveness  of  this  policy  is 
therefore  of  great  interest  to  me.  I found  the  subject  of  technology 
transfer,  and  thus  the  effectiveness  of  the  assessment  process,  to  be 
highly  complex.  I therefore  concentrated  on  six  major  issues  in  my  re- 
search. Hopefully,  these  issues  are  of  interest  to  those  who  are  involved 
with  executing  the  assessment  policy.  It  is  also  hoped  that  the  results 
of  my  research  may,  in  some  small  way,  be  of  use  either  in  executing  the 
current  policy  or  formulating  new  and  revised  policies. 

Appreciation  and  thanks  must  be  extended  to  several  people  who 
aided  me  with  their  time  and  efforts.  First,  Major  Charles  W.  McNichols, 
my  thesis  advisor,  contributed  significantly  to  this  study  in  many  ways. 
His  expertise  in  research  methods  and  his  insight  and  encouragement  were 
particularly  valuable  to  me,  I also  thank  my  reader,  Lt  Col  T.  Roger 
Manley,  for  his  assistance;  in  particular  his  assistance  in  formulating 
the  interview  questions.  I am  particularly  indebited  to  the  product  divi- 
sion and  laboratory  personnel  who  assisted  me  with  their  reviews  and  com- 
ments on  the  preliminary  interview  questions  and  draft  chapters  of  the 
thesis.  I have  chosen  not  to  name  these  individuals  in  order  to  preserve 
the  identity  of  the  product  division  and  laboratories  studied.  I wish 
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also  to  thank  Lt  CjI  Jim  Eri,  HQ  AFSC,  who  on  several  occasions  took  time 
to  assist  me  in  this  study. 

I express  my  greatest  thanks  to  Jo  Ann,  my  wife,  and  Kelli  and 
Brian,  my  children  for  their  understanding  and  encouragement  through  the 
thesis.  Thanks  also  goes  to  Jo  Ann  not  only  for  conscientious  typing  of 
the  thesis  but  for  her  assistance  in  performing  many  of  the  statistical 
computations. 

Capt  Richard  L.  Robinson 
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Abstract 

Recent  studies  conducted  by  the  United  States  Air  Force  concluded 
that  closer  ties  should  be  established  between  the  laboratories  and  product 
divisions  of  the  Air  Force  Systems  Command.  Toward  this  end,  the  Air  Force 
Systems  Command  has  established  a policy  whereby  the  product  divisions  will 
assess  advanced  development  technology  programs  conducted  by  the  labora- 
tories. This  research  effort  is  a study  to  determine  the  perceived  effec- 
tiveness of  this  assessment  policy.  Thirty-eight  structured  interviews 
were  held  with  laboratory  and  product  division  personnel.  The  perceived 
effect  of  the  assessment  policy  on  technology  transfer,  communication,  and 
dialogue,  and  the  perceived  significance  of  several  barriers  to  technology 
transfer  were  discussed  during  these  interviews.  Other  issues  discussed 
were  the  ease  or  difficulty  of  accomplishing  key  assessment  tasks,  the 
responsiveness  of  laboratories  to  product  divisions,  the  attitude  of  labor- 
atory and  product  division  personnel  with  respect  to  the  assessment  policy, 
and  the  desirability  of  encorporating  additional  technology  transfer  mech- 
anisms. The  writer  concluded  that  those  interviewed  perceived  that  the 
assessment  policy  will  tend  to  enhance  technology  transfer,  communication, 
and  dialogue.  Also,  no  significant  problems  were  perceived  to  exist  in 
accomplishijig  the  assessment  tasks.  Generally,  those  interviewed  perceived 
that  the  laboratories  will  not  tend  to  be  more  responsive  than  they  should 
be.  A favorable  attitude  toward  the  assessment  policy  was  expressed,  but 
the  additional  technology  transfer  mechanisms  were  not  strongly  endorsed. 
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ASSESSMENTS  OF  ADVANCED  DEVELOPMENT 
TECHNOLOGY  PROGRAMS:  A STUDY  TO  DETERMINE 
THE  PERCEIVED  EFFECTIVENESS  OF  A NEW 
AIR  FORCE  SYSTEMS  COMMAND  POLICY 


I.  Introduction 


In  order  to  maintain  our  most  flexible  and  imaginative 
defense  posture,  the  Department  of  Defense  (DOD)  must 
utilize  every  conceivable  resource,  capability  and  con- 
tribution it  can  possibly  motivate,  attract  or  support. 
This  requires  the  competence  and  contribution  of  all 
types  of  institutions — industry,  university,  nonprofit, 
and  in-house  organization.  . , . Each  is  an  important 
interrelated,  synergetic  subsystem  whose  products  of 
new  knowledge,  designs,  and  weaponry  are  the  first- 
line  technological  defense  against  foreseeable  threats 
Glass,  1967:1). 


The  above  statement  succinctly  characterizes  the  challenge  that 
the  Department  of  Defense  (DOD)  faces  in  maintaining  a strong  defense 
posture  based  upon  advanced  technology  weaponry.  It  is  somewhat  axiomatic 
that  success  in  meeting  this  challenge  is  heavily  dependent  upon  the  ad- 
vances in  science  and  technology  that  are  made  through  the  research  and 
development  (R&D)  programs  conducted  by  the  three  military  departments. 
However,  technical  advances  alone  are  not  sufficient.  The  "products"  of 
the  "subsystems"  mentioned  in  the  opening  quote  must  be  effectively  trans- 
ferred to  and  intergrated  with  an  end-using  weapon  system  in  order  to 
achieve  the  desired  defense  posture. 


Background 


During  the  past  fifteen  years  there  has  been  much  high  level 
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attention  paid  to  the  effectiveness  of  one  institutional  element  of  the 
military  R&D  community— the  Defense  in-hoUse  laboratories.  A persistent 
feeling  expounded  by  many  prominent  committees  and  individuals  is  that 
. . in-house  laboratories  are  the  weakest  and  least  effective  of  the 
performers  available  to  do  RDT&E  for  the  Department  of  Defense"  (Glass, 
1967:11). 

The  most  recent  review  of  the  in-house  laboratories  was  accom- 
plished in  1974  by  a joint  Office  of  the  Secretary  of  Defense  (0SD)/Mili- 
tary  Department  undertaking.  This  effort  has  become  known  as  the  Lab- 
oratory Utilization  Study  (LUS)  and  was  concerned  with  answering  questions 
regarding  the  need,  size,  and  use  of  the  in-house  laboratories.  Changes 
to  the  management  structure  and  organization  of  both  the  laboratories  and 
the  respective  headquarters  which  would  improve  the  contributions  to  the 
Department  of  Defense  (DOD)  were  also  considered  (U.  S.  Congress,  1975: 
400-402). 

Air  Force  Laboratory  Utilization  Study  Findings.  The  Air  Force 
participation  in  the  joint  study  was  in  the  form  of  a study  group  chaired 
by  Maj  Gen  Kenneth  R.  Chapman.  The  official  conclusions  and  recommendations 
reached  by  this  study  group  have  not  been  widely  disseminated  in  a final 
report.  However,  the  key  findings  and  recommendations  of  the  Air  Force 
study  group  are  highlighted  in  a summary  report  issued  by  the  Department 
of  Defense  (Allen,  1975). 

The  information  contained  in  the  Department  of  Defense  (DOD)  re- 
port indicates  that,  overall,  the  Air  Force  laboratories  were  given  a very 
positive  assessment  with  respect  to  competency  of  personnel,  responsiveness 
to  requests  for  support,  and  quality  of*  work.  The  laboratories  were  also 
judged  to  be  at  least  as  good  as  non-Air  Force  organizations  engaged  in 
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similar  work  (Allen,  1975 * 24).  These  conclusions  were  based  on  the  results 
of  a survey  conducted  in  mid-June  1974  by  faculty  members  of  the  Department 
of  Systems  Management,  Air  Force  Institute  of  Technology  (AliT).  Survey 
questionnaires  were  distributed  Air  Force-wide  to  personnel  working  in 
the  system  development  agencies.  The  questionnaires  were  designed  specif- 
ically to  measure  the  laboratory  user's  evaluation/percepcions  of  the  Air 
Force  laboratories  (Manley,  1974*1). 

Although  the  Air  Force  laboratories  were  generally  viewed  in  a 
favorable  light  by  the  system  development  community,  some  negative  obser- 
vations were  also  made.  Almost  half  of  the  survey  respondents  suggested 
ways  in  which  the  effectiveness  of  the  Air  Force  laboratories  could  be 
improved.  Their  suggestions  revealed  four  general  areas  of  concern*  (1)  an 
inadequate  Air  Force  program  for  transitioning  laboratory  developed  tech- 
nology to  the  program  offices,  (2)  inadequate  communication  between  the 
laboratories  and  the  program  offices,  (3)  inadequate  congruency  of  lab- 
oratory and  program  office  goals,  and  (4)  inadequate  visibility  of  lab- 
oratory projects  in  the  system  development  area  (Manley,  1974*1,  2). 

The  Air  Force  study  group  itself  also  suggested  ways  to  increase 
laboratory  effectiveness.  Generalizing,  the  study  group's  suggestions 
required  the  following  actions* 

1.  Continued  use  of  the  laboratories  to  train  military  officers 
in  the  research  and  development  career  field, 

2.  The  use  of  fiscal  controls  over  civilian  manpower  rather  than 
manpower  ceilings. 

3.  Adjustments  of  super  grade  civilian  positions  within  AFSC, 
more  term  appointments  for  senior  civilians.,  anu  a career 
development  plan  for  civilian  employees. 

4.  A re-examination  of  the  missions  of  the  in-house  laboratories. 

5.  Forcing  closer  ties  between  the  AFSC  laboratories  and  the  AFSC 
product  divisions  (Allen,  1975*26). 

3 


GSM/SM/76S-21 


In  order  to  achieve*  closer  ties  between  the  laboratories  and 
product  divisions  the  study  group  suggested  a realignment  of  some  of  the 
laboratories  and  recommended  that  the  product  divisions  control  the  ad- 
vanced and  engineering  development  funds  expended  by  the  laboratories. 

The  study  group  suggested  that  some  laboratories 
be*  affiliated  with  product  divisions*  AFATL  remain  with 
APTC;  AFAPL,  AFFDL,  AFAL,  and  parts  of  ARL  combine  into 
one  new  laboratory  associated  with  ASD;  a part  of  AFCRL 
(restructured),  RADG  and  RML  affiliate  with  ESD;  and 
AFRFL  with  5AMS0.  The  long  range  objective  is  to  create 
a center  of  technology  behind  each  product  division  to 
insure  Air  Force  technical  competence  in  the  key  product 
areas  of  interest.  The  AFSC  product  divisions  should 
control  the  advanced  and  engineering  development  funds 
expended  in  these  laboratories,  although  the  labora- 
tories would  continue  to  report  to  DS&T  (Allen,  1975*26). 

It  was  felt  that  product  division  control  of  advanced  and  engineering 
funds  would  "...  improve  relevancy  of  the  projects,  bring  the  labora- 
tories closer  to  system  planning  and.  acquisition,  and  provide  a direct 
link  for  the  transition  of  technology"  (Allen,  1975*25). 

The  survey  respondent  and  study  group  suggestions  identified 
above  have  resulted  in  the  formulation  of  several  new  management  policies 
by  Headquarters  Air  Force  Systems  Command  (HQ  AFSC).  One  of  these  new 
policies  requires  that  the  product  divisions  assess  advanced  development 
technology  programs  (ADTP)  conducted  by  the  laboratories  and  other  agencies. 
The  ADTP  assessment  policy  provides  the  basis  for  this  research  effort, 
and  its  implementation  is  discussed  below  in  greater  detail. 

Implementation  of  Advanced  Development  Technology  Program  (ADTP) 
Assessments.  One  of  the  first  indications  that  official  action  was  being 
considered  as  a result  of  the  Laboratory  Utilization  Study  (LUS)  was  a 
Headquarters  Air  Force  Systems  Command  (HQ  AFSC)  letter  sent  to  subor- 
dinate units  within  the  command  on  29  January  1975.  This  letter  was 
signed  by  Lt  Gen  John  B.  Hudson,  then  the  Vice  Commander  of  AFSC,  and 
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indicated  that  an  assessment  process  was  being  considered  as  a means  of 
implementing  a LUS  recommendation  to  increase  the  responsiveness  of  the 
laboratories  to  the  product  divisions.  The  desired  result  of  the  assess- 
ment process  proposed  was  to  "improve  the  transition  of  technology"  (HQ 
AFSC  (CV)  letter,  29  Jan  1975iPar  1).  The  letter  also  indicated  that 
the  information  collected  by  the  assessments  would  be  used  to  establish 
an  AFSC  position  on  those  advanced  and  engineering  development  efforts  that 
should  be  pursued.  The  specific  assessment  process  outlined  in  that  letter 
is  as  follows: 

...  It  is  envisioned  that  the  Laboratories  will 
annually  submit  6. 3/6.4  program  documentation  to  AFSC 
through  the  appropriate  Product  Division(s).  The  Pro- 
duct Division  will  add  an  assessment  of  relevance, 
cost  implications,  timeliness,  payoff,  and  relative 
priority  of  proposed  efforts,  both  new  starts  and 
continuing  programs  (L'Q  AFSC  (CV)  letter,  29  Jan  1975: 

Par  2). 

The  letter  was  not  directive  in  nature  but  was  replete  with  qualifiers 
indicating  that  the  overall  process  for  conducting  the  assessments  was 
not  firm.  Suggestions  for  the  actual  implementation  were  solicited  from 
the  various  laboratories  and  product  divisions. 

Considerable  interaction  between  HQ  AFSC,  the  product  divisions, 
and  the  laboratories  followed  Gen  Hudson's  letter.  "Srainstorming" 
sessions  were  held,  test  case  assessments  were  completed,  and  a "quick 
response"  prioritization  of  advanced  development  efforts  by  all  of  the 
product  divisions  was  forwarded  to  HQ  AFSC.  The  experience  gained  from 
these  activities  lead  to  the  development  of  a definitive  set  of  procedures 
for  the  assessment  process.  The  procedures  and  the  requirement  to  imple- 
ment the  assessment  process  were  officially  promulgated  by  the  new  Vice 
Commander  of  AFSC,  Lt  Gen  Robert  T.  Marsh,  on  5 December  1975  (HQ  AFSC 
(CV)  letter,  5 Dec  1975).  These  procedures  have  subsequently  been  published 
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as  AFSCR  80-19,  This  document  as  well  as  Gen  Hudson's  ar.d  Gen  Marsh's 
letters  a re  reproduced  in  Appendix  B for  the  convenience  of  the  reader. 

The  ADTP  assessment  process  clearly  represents  management  action 
taken  to  alleviate  problems  regarding  inadequate  responsiveness  of  the 
laboratories  to  the  product  divisions,  inadequate  dialogue  between  the 
laboratories  and  product  divisions,  and  inadequate  transfer  of  technology. 
These  factors  had  been  perceived  as  problem  areas  by  Air  Force  management 
prior  to  the  Laboratory  Utilization  Study  (LUS),  and  management  action  was 
also  taken  at  that  time. 

Previous  Management  Action.  In  1971  the  Air  Force  submitted  a 
plan  of  action  to  the  Task  Group  on  In-house  Laboratories,  and  this  plain 
contained  several  management  improvement  actions  that  had  either  been 
completed  or  were  under  consideration  at  that  point  in  time  (ODDR&E, 
1971:57-115). 

One  of  these  management  improvement  actions  was  the  conversion 
of  the  Research  and  Technology  Division  (RTD)  to  the  Director  of  Labora- 
tories (DOL)  organization  in  November  1966.  The  Air  Force  felt  that  this 
conversion  would  provide  the  following  benefits: 

a.  Increased  relevance  of  laboratory  work  to  system 
development  and  operational  requirements. 

b.  Improved  participation  of  laboratories  in  system 
planning  and  concept  formulation, 

c.  Greater  laboratory  responsiveness  to  system  devel- 
opment needs  and  technical  solution  of  operational  prob- 
lems . . . (QPDR&E,  1971:79). 

These  expected  benefits  clearly  indicate  that  prior  to  establishing  the 
Director  of  Laboratories  organization  inadequate  relevance  of  laboratory 
work,  inadequate  laboratory  participation  in  system  planning,  and  inade- 
quate laboratory  responsiveness  to  system  development  needs  had  been 
perceived  as  problem  areas  by  Air  Force  management. 
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Another  action  mentioned  vas  the  establishment  of  coupling  pro- 
cedures. The  Technology  Need  program,  established  in  1963,  was  cited  as 
providing  the  procedures  for  technology  coupling  (ODDR&E,  1971:109).  Addi- 
tionally, the  Air  Force  indicated  that  procedures  for  obtaining  laboratory 
direct  support  were  documented  in  existing  HQ  AFSC  Regulations.  Two  of 
the  benefits  expected  to  be  received  from  these  procedures  were  increased 
dialogue  between  the  laboratories  and  the  product  divisions  and  more  face 
to  face  interaction  (ODDR&E,  1971:109).  These  expected  benefits  clearly 
indicate  that,  prior  to  establishing  the  above  procedures,  inadequate 
coupling  of  technology  and  inadequate  dialogue  between  the  laboratories 
and  product  divisions  had  also  been  perceived  as  problem  areas  by  Air 
Force  management. 

Statement  of  the  Problem 

The  fact  that  the  previous  management  actions  enumerated  above 
addressed  the  same  problem  areas  that  the  ADTP  assessment  process  addresses 
is  clear.  It  is  felt  that  the  persistence  of  these  problem  areas  demon- 
strates the  difficulty  encountered  in  achieving  a desired  solution.  Thus, 
the  logical  question  becomes,  "How  effective  will  the  ADTP  assessment 
process  be  in  alleviating  these  difficult  problem  areas?" 

The  problem  perceived  by  the  researcher  was  to  obtain  some  measure 
of  the  true  effectiveness  of  the  ADTP  assessment  process.  This  proved  to 
be  impossible  because  the  process  had  just  been  implemented  and  data 
which  could  be  used  to  explicitly  measure  success  or  failure  did  not 
exist.  Thus,  as  a surrogate  of  true  effectiveness,  it  was  decided  to 
attempt  to  measure  the  perceived  effectiveness  of  the  process  by  obtaining 
feedback  from  laboratory  and  product  division  personnel  who  have  been  in- 
volved in  implementing  and  executing  the  assessment  policy. 
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Scope  and  Limitations 

Currently,  there  are  both  proponents  and  opponents  of  the  ADTP 
assessment  process.  The  objective  of  this  thesis  does  not  include  a con- 
clusion as  to  whether  this  process  is  either  good  or  bad.  Rather  this 
study  centers  on  determining  the  perceived  effectiveness  of  the  process 
with  respect  to  alleviating  the  problem  areas  identified  above.  The 
study  addresses  the  following  specific  issues* 

1.  What  is  t»!e  perceived  effect  of  the  ADTP  assessment,  process 
on  the  transfer  of  technology? 

2.  What  is  the  perceived  effect  of  the  ADTP  assessment  process 
on  communication  and  dialogue? 

3.  What  is  the  relative  ease  or  difficulty  of  accomplishing 
key  tasks  required  by  the  assessment  process? 

A.  Will  the  laboratories  be  as  responsive  to  the  product  divi- 
sions as  they  should  be? 

5.  What  is  the  general  attitude  of  product  division  and  lab- 
oratory personnel  with  respect  to  the  assessment  process? 

6.  Do  additional  technology  transfer  mechanisms  formulated  by  the 
researcher  have  enough  merit  to  warrant  further  consideration? 

Ideally  a study  of  this  nature  would  consider  all  product  divi- 
sions and  all  laboratories,  but  the  effort  required  for  such  a comprehen- 
sive study  far  exceeded  the  time  available  for  work  on  the  thesis.  Con- 
sequently, the  study  was  purposefully  limited  to  one  product  division  and 
three  related  laboratories.  This  limitation  in  scope  dictates  the  use  of 
caution  when  extrapolating  the  conclusions  and  findings  of  the  study  to 
all  product  divisions  and  laboratories. 

A final  limitation  on  this  research  effort  is  the  source  of  data. 

A great  deal  of  information  was  gathered  through  personal  interviews  and 
letters.  Since  a promise  of  anonymity  was  paramount  in  securing  some  of 
this  information,  neither  the  p^ndnet  division  nor  laboratories  visited 


will  be  identified. 
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Assumptions 

One  of  the  underlying  assumptions  required  by  this  research  effort 
is  that  the  personnel  interviewed  truthfully  expressed  their  attitude  and 
opinions  in  responding  to  the  questions  asked  since  their  anonymity  was 
assured.  The  researcher  perceived  this  to  be  in  fact  true,  and  it  is  felt 
that  a review  of  the  subjective  comments  listed  in  Appendix  E will  support 
the  validity  of  this  assumption. 

Other  assumptions  required  for  the  study  are  those  imposed  by  the 
statistical  tests  used  during  the  analysis  of  the  data.  These  include  the 
assumptions  that  the  interviews  conducted  represent  a random  sample  and 
that  responses  obtained  from  those  interviewed  are  independent.  The  va- 
lidity  of  these  assumptions  is  discussed  in  the  chapter  on  research  meth- 
odology (Chapter  III). 


Objectives 

The  specific  objectives  of  this  study  aret 

1.  To  provide  empirical  evidence  which  characterizes  the  per- 
ceived effectiveness  of  the  AhTP  assessment  process. 

2.  Determine  if  the  effectiveness  of  the  assessment  process  per- 
ceived by  the  various  individuals  interviewed  is  dependent 
upon  demographic  variables,  such  as  the  individual’s  organ- 
ization, grade,  time  of  service  in  the  field  of  research 

and  development,  or  time  in  the  present  job  assignment. 

3.  Identify  problem  areas  which  may  be  encountered  in  executing 
the  assessment  process  and  recommend  possible  solutions. 

4.  Determine  the  general  attitudes  of  personnel  working  in  the 
product  divisions  and  laboratories  with  respect  to  the  assess- 
ment process, 

5.  Explore  the  desirability  of  incorporating  additional  technology 
transfer  mechanisms,  such  as  laboratory /product  division  rating 
schemes,  making  future  funding  contingent  upon  past  transfers 

of  technology,  and  the  creation  of  a technology  transfer  agency. 

The  ultimate  goal  of  the  research  effort  is  to  determine  the 
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effectiveness  of  the  ADTP  assessment  policy  as  perceived  by  those  lab- 
oratory and  product  division  personnel  who  are  charged  with  its  imple- 
mentation and  execution.  It  is  hoped  that  findings  in  the  areas  inves- 
tigated may  be  of  some  value  to  Air  Force  management  personnel  in  terms 
of  either  formulating  revised  policies  or  in  effectively  administering 
existing  ones. 

Conduct  of  the  Research  (Overview) 

This  research  effort  consisted  of  four  phases  of  activity:  for- 
mulating a schedule  of  interview  questions,  conducting  formal  interviews 
with  personnel  responsible  for  executing  the  assessment  policy,  reducing 
and  analyzing  the  data  collected  during  the  interviews,  and  synthesizing 
conclusions  and  recommendations.  The  schedule  of  interview  questions  was 
developed  through  a process  of  question  formulation  and  review.  Infor- 
mation obtained  through  an  extensive  search  of  relevant  literature  and 
comments  received  from  key  personnel  assigned  to  HQ  AFSC,  one  product 
division,  and  two  laboratories  were  used  to  formulate  the  final  set  of 
interview  questions.  This  final  set  of  interview  questions  is  shown  in 
Appendix  C,  and  a discussion  of  the  individual  questions  can  be  found  in 
the  chapter  on  research  methodology.  The  formal  interviews  were  conducted 
during  a three-week  period  from  29  April  1976  to  20  May  1976.  In  all, 
thirty-eight  people  were  interviewed.  These  personnel  were  assigned  to 
the  product  division,  the  laboratories,  HQ  AFSC,  and  an  operational  com- 
mand liaison  office.  Both  subjective  comments  and  answers  to  the  specific 
questions  were  obtained  from  those  interviewed.  The  selection  of  the  per- 
sonnel interviewed  is  discussed  in  the  chapter  on  research  methodology. 

The  responses  to  the  interview  questions  are  summarized  in  Appendix  D, 
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and  elected  excerpts  from  the  subjective  comments  received  during  the 
interviews  are  listed  in  Appendix  E. 

The  analysis  of  the  data  collected  during  the  interviews  is  struc- 
tured to  address  the  specific  issues  enumerated  above  in  "Scope  and  Limita- 
tions" and  to  achieve  the  specific  research  objectives.  Both  the  results 
obtained  through  the  application  of  several  analytical  techniques  to  the 
interview  data  and  the  subjective  comments  received  during  the  interviews 
were  considered  in  formulating  the  final  conclusions.  The  specific  details 
of  the  analysis  are  presented  in  Chapter  IV,  and  conclusions  and  recommen- 
dations are  presented  in  Chapter  V. 

Chapter  Summary 

Much  high  level  attention  has  been  directed  toward  the  effec- 
tiveness of  the  Defense  in-house  laboratories.  The  most  recent  review 
of  these  agencies  was  accomplished  in  1974  by  a joint  Office  of  the  Sec- 
retary of  Defense  (OSD)/Military  Department  study  known  as  the  Laboratory 
Utilization  Study  (LUS),  Several  suggestions  were  made  by  the  Air  Force 
study  group  participating  in  the  LUS,  and  one  of  these  suggestions  was  to 
fome  closer  ties  between  the  AFSC  product  divisions  and  the  AFSC  lab- 
oratories. This  suggestion  for  closer  ties  resulted  in  the  formulation 
of  a new  management  policy  by  HQ  AFSC.  The  policy  implemented  requires 
the  product  divisions  to  assess  advanced  development  technology  programs 
(ADTP)  conducted  by  the  laboratories  and  other  agencies. 

Previous  to  the  LUS,  the  Air  Force  had  taken  management  action 
designed  to  alleviate  problem  areas  similar  to  those  perceived  by  the 
LUS  group:  inadequate  relevancy  of  laboratory  programs,  inadequate  dia- 
logue between  the  laboratories  and  the  product  divisions,  and  inadequate 
transfer  of  technology  to  system  application.  The  persistence  of  these 
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problems  characterizes  the  difficulty  associated  with  achieving  a desired 
solution.  Thus,  the  researcher  decided  to  attempt  to  measure  the  perceived 
effectiveness  of  the  ADTP  assessment  process  with  respect  to  alleviating 
these  difficult  problems. 

The  problem  of  measuring  the  perceived  effectiveness  is  addressed 
through  the  conduct  of  structured  interviews  with  personnel  responsible 
for  executing  the  policy.  In  all,  thirty-eight  personnel  were  interviewed. 
Chapter  II  follows  and  is  an  overview  of  several  concepts  related  to  the 
research  problem. 
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II.  Overview  of  Key  Concepts 

This  chapter  is  an  overview  of  several  key  concepts  which  are 
related  to  the  research  problem!  the  structure  of  the  research  and  devel- 
opment (R&D)  program  of  the  Air  Force,  the  scope  of  product  division  and 
laboratory  efforts,  the  assessment  of  advanced  development  technology 
programs,  and  related  research  on  the  transfer  of  technology. 

R&D  Program  Structure 

The  research  and  development  (R&D)  program  of  the  Air  Force  is 
divided  into  five  categories!  research,  exploratory  development,  advanced 
development,  engineering  development,  and  management  and  support.  These 
categories  of  activity  are  differentiated  by  a consideration  of  the  level 
of  technical  knowledge  existing  in  the  field  of  investigation,  the  scope 
and  thrust  of  the  research  and  development  effort,  and  the  means  of  pro- 
gram control  and  funding.  Moreover,  each  R&D  category  is  subdivided  into 
program  elements.  A program  element  may  contain  a number  of  projects  in 
a related  field,  or  it  may  represent  a single  major  project.  Funds  are 
generally  allocated  at  the  program  element  level  (AFR  80-1,  1970i2). 

Each  program  element  is  assigned  a six-digit  number  based  on  the 
DOD  programming  and  budgeting  system.  The  structure  of  this  program 
element  number  is  shown  in  Figure  1.  The  first  digit  represents  the  DOD 
major  force  program.  The  second  digit  corresponds  to  the  category  of 
activity  within  that  DOD  program.  The  remaining  three  digits,  in  com- 
bination, identify  a specific  program  element  or  project  (AFR  80-1,  1970i5j 
00C,  1975i22-28),  Often  the  first  two  digits  of  the  program  element  number 
are  used  synonymously  with  the  R&D  categories.  In  such  cases  6.1  corresponds 
to  "research,"  6.2  corresponds  to  "exploratory  development,"  and  so  on. 
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COMPONENT 
A Army 
N Navy 
M Marine  Corps 
■ F . Air  Force 


SERIAL  NUMBER * 

In  combination  with  the  second  and 
third  letter,  identifies  the  specific 
program  element 
02-ABRES 
04-AIM-40  Growth 


BUDGET  ACTIVITY * 

0 = HAD  Support  from  Other  Appropriations 
1 = Research  (Military  Sciences) 

2=  Aircraft  and  Related  Equipment 
8 *=  Missiles  and  Related  Equipment 
Military  Astronautics 

5 as  Ships  and  Small  Craft 

6 ■■  Ordnance  and  Combat  Vehicles 

7 as  Other  Equipment 

8 as  Management  and  Support 


CATEGORY • 

1.  Research 

2.  Exploratory  Development 

3.  Advanced  Development 
-►  4.  Engineering  Development 

8.  Management  and  Support 

PROGRAM 

1.  Strategic  Forces 

2.  General  Purpose  Forces 

3.  Intelligence  and  Communications 

4.  Airlift/Sealift 

5.  Guard  and  Reserve  Forces 

' 4.  Research  and  Development 

7.  Central  Supply  and  Maintenance 

8.  Training,  Medical  and  other  General  Personnel  Activities 

9.  Administration  and  Associated  Activities 
t.  Support  of  other  Nations 


*ApaBm  to  tto  USD  Pnma 


Figure  1,  Program  Element  Number  Structure  (AFR  80-1,  1970:5) 


GSM/SM/76S-21 


1 


Research.  This  program  category  represents  the  genesis  of  the 
research  and  development  cycle.  The  activities  conducted  within  this 
category  are  directed  towards  increasing  fundamental  knowledge  and  under- 
standing in  the  fields  of  science  related  to  national  security  needs. 

This  knowledge  and  understanding  contributes  to  the  technology  base  re- 
quired for  subsequent  exploratory  and  advanced  development  efforts,  but 
the  solution  to  a specific  military  problem  is  generally  not  identified 
(AFR  80-1,  1970:2).  Policy  guidance  for  research  efforts  is  provided  by 
HQ  USAF,  and  the  Director  of  Science  and  Technology,  HQ  AFSC  (DL),  is  re- 
sponsible for  allocating  the  funds  to  specific  program  elements  (AFSCR  23-1, 


1975:4-60). 


tory  Development.  Unlike  research,  exploratory  development 


activities  are  directed  toward  the  solution  of  a specific  military  problem. 
They  are  designed  to  provide  proof  of  the  technical  feasibility  of  a new 
concept  in  terms  of  a military  application  and  to  reduce  the  risk  associated 
with  further  development  if  the  new  idea  appears  promising.  Generally, 
only  one  new  technology  is  addressed  by  a specific  program  or  project. 

Policy  guidance  on  exploratory  development  efforts  are  provided 
by  HQ  USAF,  and  the  Director  of  Science  and  Technology,  HQ  AFSC  (DL),  is 
responsible  for  allocating  funds  to  spec  fie  program  elements  (AFSCP.  23-1, 
1975:4-60).  Normally,  financial  control  is  exercised  by  level  of  effort 
funding  since  this  provides  flexibility  to  the  laboratory  director  in  al- 
locating resources  (AFR  80-1,  1970:2), 

Advanced  Development.  One  distinguishing  feature  of  an  advanced 
development  effort  is  the  development  of  hardware  for  experimentation  or 
test  rather  than  actual  service  use.  In  theory,  advanced  development  pro- 
grams are  used  to  prove  either  a new  subsystem  or  technology  before  these 
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new  items  are  used  in  an  actual  weapon  system  (AFSCR  27-6,  1974:5-20). 

The  intent  is  to  build  a confidence  level  sufficient  to  warrant  further 
development  of  the  new  capability  if  the  requirement  and  funds  remain 
available. 

Advanced  development  programs  (ADPs)  generally  require  a greater 
expendature  of  R&D  resources  than  either  research  or  exploratory  develop- 
ment efforts.  Thus  the  allocation  of  funds  to  specific  program  elements 
is  made  by  HQ  USAF  rather  than  the  Director  of  Science  and  Technology. 

Also,  monitor  and  control  of  advanced  development  efforts  are  accomplished 
on  a project  by  project  basis  rather  than  by  the  level  of  effort  basis 
commonly  used  for  research  and  exploratory  development  (AFR  80-1,  1970:2). 

Engineering  Development.  A program  is  classified  in  the  engi- 
neering development  category  if  the  item  being  developed  is  intended  For 
actual  service  use  (AFR  80-1,  1970:2).  These  activities  occur  near  the 
end  of  the  research  and  development  cycle.  At  this  ptoint  in  tim^  the 
technological  unknowns  have  been  virtually  eliminated,  and  the  primary 
effort  is  directed  to  the  system  engineering  and  testing  required  to 
convert  the  knowledge  and  equipment  gained  from  the  earlier  phases  of 
R&D  into  a new  weapxm  system.  Engineering  development  program  elements 
are  subdivided  into  projects,  and  the  programming  and  budgeting  activ- 
ities are  accomplished  at  the  project  level  by  HQ  USAF  (AFSCR  27-6, 

1974:5-21). 

Management  and  Support.  This  category  of  research  and  developj- 
ment  contains  program  elements  for  engineering  support  from  several  Federal 
Contract  Research  Centers  (FCRC),  such  as  MITRE,  RAND,  and  the  Aerospace 
Corporation.  Program  elements  which  provide  for  the  operating  cost  of  AFSC- 
operated  facilities  and  test  ranges  are  also  included  (AFSCR  27-6,  1974:5-21). 
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Scope  of  Product  Division  and  Laboratory  Efforts 

The  product  divisions  and  laboratories  are  responsible  for  dif- 
ferent activities.  Typically,  the  product  divisions  are  responsible  for 
the  later  categories  of  R&D  effort,  and  the  laboratories  are  responsible 
for  the  earlier  phases.  A brief  review  of  the  scope  of  these  efforts  will 
provide  information  supportive  to  the  discussion  of  the  technology  transfer 

concepts  presented  later.  Table  I lists  the  agencies  considered  as  product 
* 

divisions  and  laboratories  during  the  course  of  this  research  effort.  How- 
ever, it  should  be  noted  that  personnel  from  only  one  product  division  and 
three  laboratories  were  actually  interviewed  during  the  conduct  of  this 
research  effort. 

Product  Division.  The  product  divisions  are  officially  recognized 
as  AFSC  field  commands.  The  precursors  to  the  agencies  existing  today  were 
formed  through  several  organizations  and  reorganizations  of  the  Air  Force 
R&D  elements  during  the  time  period  between  November  1959  and  the  estab- 
lishment of  the  Air  Force  Systems  Command  (AFSC)  in  April  1961  (NSF  62-37, 
1963t255). 

The  scope  of  the  product  division  effort  is  very  broad  in  that 
they  are  responsible  for  the  management  of  the  acquisition  of  new  weapon 
systems.  A discussion  of  this  system  acquisition  process  is  well  beyond 
the  intended  scope  of  these  overview  remarks.  Rather  the  intent  is  to 
provide  an  indication  of  the  categories  of  R&D  activity  that  the  product 
divisions  typically  conduct.  The  reader  interested  in  the  management  of 
the  acquisition  process  is  referred  to  AFSCP  800-3. 

Any  generalization  of  the  R&D  act:'  - ..ties  conducted  by  product 
divisions  is  doomed  to  be  in  error  because  the  system  acquisition  and 
R&D  processes  are  highly  interwoven  and  complex.  However,  it  can  be 
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stated  with  some  degree  of  accuracy  that  the  product  divisions  primarily 
conduct  R&D  efforts  in  the  engineering  development  category.  These  efforts 
are  associated  with  validation  and  full  scale  development  phases  of  weapxtn 
system  acquisition,  and  examples  include  the  development  of  items  such  as 
the  B-l  and  F-16  prototype  aircraft.  System  related  advanced  development 
efforts  are  also  conducted,  but  the  magnitude  of  these  efforts  is  consid- 
erably less  than  that  of  the  engineering  development  activities.  The 
product  divisions  rely  almost  exclusively  on  industry  contractors  to  ac- 
complish the  actual  development  work  (AFSCP  800-3,  1976). 

Laboratory . The  laboratories  are  responsible  for  two  broad 
classes  of  effort:  development  of  the  technology  base  and  support  to 
other  Air  Force  agencies.  The  laboratories  support  the  technology  base 
by  serving  as  an  Air  Force  interface  with  the  technological  community 
and  by  conducting  research,  exploratory  development,  and  advanced  devel- 
opment programs.  Support  to  other  agencies  is  generally  provided  in 
the  form  of  technical  assistance.  This  includes  assistance  in  solving 
technical  problems  encountered  during  weapon  system  acquisition  by  the 
product  divisions  and  in  the  operational  environment  by  the  operational 
commands.  The  laboratories  also  support  higher  headquarters  by  providing 
technical  information  for  long  range  military  planning  and  decision  making 


(AFR  80-3,  1971:2). 


The  bulk  of  the  laboratory  effort  is  devoted- to  suppxnrting  the 
technology  base.  Approximately  half  of  the  funds  available  to  the  lab- 
oratories come  from  the  research  (6.1)  and  exploratory  development  (6.2) 
budget  categories,  mostly  the  latter.  About  35%  of  the  funds  comes  from 
the  other  RDT&E  budget  categories,  primarily  advanced  development  (6,3), 
and  the  remaining  15%  comes  from  non-RDT&E  budget  categories.  This  latter 
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source  of  funding  represents  the  amount  of  effort  devoted  to  the  support 
of  existing  weapon  systems  and  other  activities  (Allen,  1975t3). 

The  laboratories  rely  on  industry  contractors  for  the  conduct 
of  much  of  the  actual  development  work.  Only  30%  of  the  research  and 
exploratory  development  efforts  and  only  5%  of  the  total  RDT&E  effort  are 
actually  performed  in-house  (Allen,  1975t3). 

Assessments  of  Advanced  Development  Technology  Programs 

The  background  and  implementation  of  the  ADTP  assessment  require- 
ment was  presented  in  the  introductory  remarks  of  Chapter  I,  Moreover, 
the  scope,  anticipated  benefits,  and  a detailed  specification  for  the 
assessment  paper  are  described  in  the  implementing  regulation,  AFSCR  80-19 
(see  Appendix  B).  The  ensuing  discussion  is  a summary  of  the  assessment 
process  for  new  and  continuing  programs  and  a brief  overview  of  the  rela- 
tionship of  the  assessments  to  the  formulation  of  an  AFSC  advocacy  posi- 
tion regarding  the  conduct  of  research  and  development  programs  within 
the  laboratories. 

However,  before  the  discussion  proceeds,  the  source  of  the  in- 
formation presented  requires  clarification.  The  only  official  sources 
of  information  available  were  Gen  Marsh's  5 December  letter  and  AFSCR  80-19 
(see  Appendix  B).  The  remaining  information  was  obtained  through  informal 
interviews  with  personnel  working  within  HQ  AFSC,  the  product  division, 
and  the  laboratories,  as  well  as  from  briefing  charts  provided  by  these 
individuals.  In  fact,  except  for  format,  the  figures  presented  are  dup- 
licates of  briefing  charts  obtained  during  the  visit  to  HQ  AFSC. 

Assessment  of  New  and  Continuing  Programs.  An  overview  of  the 
ADTP  assessment  process  and  subsequent  program  direction  is  shown  for  new 
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new  programs  in  Figure  2 and  continuing  programs  in  Figure  3.  A compar- 
ison of  these  two  illustrations  reveals  that  the  salient  features  of 
the  assessment  process  for  these  two  types  of  programs  are  similar.  The 
laboratories,  in  responding  to  official  guidance,  prepare  documentation 
which  is  concurrently  forwarded  to  HQ  AFSC  (DL)  and  the  appropriate  prod- 
uct divisions.  The  product  divisions  forward  completed  assessments  to 
HQ  AFSC  (XR).  After  coordination  within  HQ  AFSC,  an  advocacy  position 
is  presented  to  the  Air  Staff. 

Program  direction  for  authorized  programs  is  provided  from  HQ  USAF 
to  HQ  AFSC  in  the  Program  Management  Directive  (IMD).  HQ  AFSC  (DL),  in 
turn,  provides  program  direction  to  the  appropriate  laboratories  through 
the  use  of  an  AFSC  Form  56. 

The  primary  differences  in  the  application  of  the  assessment 
policy  between  the  new  programs  and  continuing  programs  is  in  the  details 
of  the  planning  documentation  prepared  by  the  laboratories,  the  depth  of 
the  assessments  prepared  by  the  product  divisions,  and  the  timing.  At 
the  time  of  this  writing  the  official  position  regarding  the  details  of 
the  information  required  in  the  proposed  development  plan,  referred  to 
as  a Technology  Program  Plan  (TPP)  in  AFSCR  80-19,  is  being  coordinated 
within  HQ  AFSC  (HQ  AFSC  (DLX)  letter,  21  April  1976),  Conceptually,  how- 
ever, it  will  contain  sufficient  information  for  the  product  divisions  to 
perform  the  required  initial  in-depth  assessment  with  respect  to  payoff, 
relevance,  cost  implications,  timeliness,  and  relative  priority.  More 
detail  on  the  specific  information  required  in  the  initial  in-depth  assess- 
ment paper  is  contained  in  Attachment  1 to  AFSCP,  80-19  (see  Appendix  B). 

The  assessment  of  current  programs  is  an  annual  requirement.  The 
laboratories  provide  updated  planning  documents  and  planning  summaries  to 
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HQ  AFSC  (DL)  and  the  appropriate  product  divisions.  The  product  divisions 
cither  confirm  or  modify  the  previous  assessments  and  the  relative  priority 
assigned  and  provide  this  ?. -formation  to  HQ  AFSC  (XR).  The  specific  infor- 
mation required  in  these  annual  assessments  is  contained  in  Attachment  2 
to  AFSCR  80-19  (see  Appendix  B). 

Program  direction  for  programs  authorized  to  continue  is  essen- 
tially the  sane  as  that  of  new  programs.  The  noticeable  difference  is  the 
use  of  revised  program  direction  documentation  in  lieu  of  new  documentation. 

Support  for  the  AFSC  Advocacy  Position.  Within  the  last  two 
years  new  ideas  and  procedures  have  been  implemented  in  order  to  assist 
in  the  development  of  an  HQ  ?-FSC  advocacy  position  regarding  which  research, 
exploratory  development,  and  advanced  development  program  should  be  pursued. 
The  assessment  of  ADTPs  and  the  formulation  of  an  AFSC  investment  strategy 
(Status  Report,  March  1976)  are  two  of  these  new  ideas.  The  key  aspects 
of  the  process  followed  in  developing  the  advocacy  position  for  the  fiscal 
year  (FY)  1978  and  the  fiscal  year  (FY)  1979  programs  are  shown  in  Figure  4. 

The  formulation  of  the  advocacy  position  began  in  November  1975 
with  the  establishment  of  development  goals  by  the  Deputy  Chief  of  Staff 
for  Development  Plans,  HQ  AFSC  (XR),  The  assessment  activity  identified 
in  the  subsequent  two  blocks  shown  in  Figure  4 was  devoted  to  current  pro- 
grams and  projects  (FY  1978/FY  1979),  AFSC  (XR)  concentrated  on  advanced 
development  technology  programs  (6.3),  and  the  Erector  of  Science  and 
Technology,  HQ  AFSC  (DL),  concentrated  on  exploratory  development  pro- 
grams (6.2). 

The  primary  function  of  the  Joint  Planning  Group  (JPG)  was  to 
develop  the  proposed  investment  strategy.  The  group  met  during  the 
January-February  1976  time  period,  and  its  membership  was  comprised  of 
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personnel  from  AFSC  (DL),  AFSC  (XR),  and  the  laboratories  and  product 
divisions  identified  in  Table  I (Status  Report,  March  1976).  The  as- 
sessment papers  prepared  by  the  product  divisions  during  the  time  period 
between  June  1975  and  Janu?^ry  1976  addressed  programs  to  be  conducted 
during  FY  1977  and  FY  1978,  These  assessments  were  available  for  con- 
sideration by  the  Joint  Planning  Group  in  developing  the  proposed  in- 
vestment strategy. 

The  initial  formulation  of  the  AFSC  investment  strategy  was  com- 
pleted in  February  1976.  Final  formulation,  as  shown  in  Figure  4,  involved 
interaction  with  higher  echelons  of  management  within  the  Air  Force  and 
the  Department  of  Defense  to  discuss,  resolve,  and  incorporate  various 
issues  and  alternatives. 

Once  finalized,  the  official  investment  strategy  was  used  by 
AFSC  (DL)  and  AFSC  (XR)  as  an  input  to  the  development  of  planning  guidance 
for  laboratories  and  product  divisions.  The  guidance  provided  the  basis 
for  formulating  either  new  programs  or  revisions  to  current  programs  within 
the  laboratories.  The  new  advanced  development  technology  programs  pro- 
posed would  be  those  scheduled  to  start  in  FY  1979. 

The  planning  documentation  for  these  programs  will  be  forwarded 
to  AFSC  (DL)  and  the  appropriate  product  divisions  by  December  1976.  These 
programs  will  be  assessed  by  the  product  divisions  as  discussed  earlier. 
Once  the  assessments  are  completed  and  forwarded  to  AFSC  (XR),  a joint 
AFSC  (DL)  and  AFSC  (XR)  advocacy  position  will  be  formulated.  This  posi- 
tion will  be  used  to  suppoi.~.  the  budgeting  activity  for  FY  1978,  the 
planning,  programming,  and  budgeting  activities  for  FY  1979,  and  the  de- 
velopment of  goals  for  FY  1979,  Once  new  development  goals  are  formulated 
the  cycle  is  repeated. 
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Related  Research  on  the  Transfer  of  Technology 

Improving  the  transfer  of  technology  is  explicitly  stated  as  one 
of  the  anticipated  benefits  of  the  ADTP  assessment  process.  The  litera- 
ture search  conducted  as  part  of  this  research  effort  revealed  that  the 
subject  of  technology  transfer  has  been  studied  by  both  governmental 
agencies  and  interested  individuals.  Moreover,  many  of  these  research 
efforts  have  identified  specific  factors  which  appear  to  either  aid  or 
represent  a barrier  to  the  transfer  of  technology. 

Two  studies  which  identified  factors  that  appear  to  aid  the 
transfer  of  technology  were  sponsored  by  govermental  agencies.  In  1969 
the  Department  of  Defense  completed  Project  Hindsight.  This  project  was 
designed  to  study  research  and  technology  advances  that  have  been  used 
in  weapon  systems  (ODDR&E,  1969:135).  Also,  in  1974  the  National  Science 
Foundation  (NSF)  sponsored  a study  by  the  Syracuse  University  Research 
Corporation  to  investigate  the  transfer  of  technology  from  federal  lab- 
oratories to  commercial  use  (Teich,  1974). 

The  National  Science  Foundation  has  also  sponsored  studies  to 
identify  factors  that  appear  to  be  barriers  to  the  transfer  of  technology. 

Dr,  Joseph  Gartner  and  Dr.  Charles  S.  Naiman  synthesized  the  results  from 
several  of  these  studies  and  formulated  a set  of  specific  barriers  (Gartner, 
1976:23). 

This  research  effort  draws  upon  the  results  of  all  of  the  above 
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studies.  Several  interview  questions  were  developed  by  considering  the 

'A 

various  factors  which  were  identified  as  either  aids  or  barriers  to  the  1 

transfer  of  technology.  Accordingly,  the  following  discussion  is  an  over-  < 

l 

view  of  the  principal  findings  and  observations  of  the  above  studies  that 
were  used  in  this  research  effort.  The  development  of  specific  interview 
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questions  is  discussed  more  thoroughly  in  the  chapter  on  research  meth- 
odology. 


Project  Hindsight.  This  project  was  established  by  the  Director 
of  Defense  Research  and  Engineering  in  June  1965  with  the  objective  of 
conducting  a comprehensive  analysis  of  the  impact  of  rer - arch  and  tech- 
nology on  a number  of  weapon  systems  (ODDR&E,  1969:135).  The  resulting 
analysis  was  indeed  comprehensive.  Twenty  different  weapon  systems  were 
reviewed,  and  710  research  and  exploratory  development,  events  which  had 
contributed  to  advancing  the  technology  used  in  these  weapon  systems  were 
investigated  in  detail. 

One  "strategy''  pursued  by  the  Project  Hindsight  researchers  was 
to  "determine  significant  management  and  other  enviro.imental  factors,  as 
seen  by  the  research  scientist  or  engineer,  that  appear  to  be  commensurate 
with  high  utilization  of  research  results"  (ODDR&E,  1969:9).  Several  such 
factors  were  identified  as  principal  findings  by  the  Project  Hindsight 
study  team,  and  those  which  were  useful  in  formulating  interview  questions 
are  summarized  below? 

1.  The  most  utilized  new  scientific  information  comes  from  pro- 
grams organized  in  response  to  a recognized  DOD  problem 
(ODDR&E,  1969:40). 

2.  In  over  85%  of  the  technological  events  studied,  the  orig- 
inal recognition  of  the  problem  or  need  was  made  by  the 
applications  engineering  group.  However,  the  solution  to 
the  problem  was  found  by  the  research  group  in  72%  of  these 
events  (ODDR&E,  1969:47). 

3.  "The  dominant  communication  path  for  the  transfer  of  tech- 
nological information  has  bean  informal,  person  to  person 
contact"  (ODDR&E,  1969:48), 

4.  "The  engineer  appears  to  rely  heavily  on  unified  scientific 
theory  and  tabulated  scientific  information  published  in 
engineering-data  handbooks  and  text  reference"  (ODDR&E, 
1969:41). 
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Syracuse  University  Research  Corporation  Study,  This  research 
effort  encompassed  eleven  different  case  studies  of  the  transfer  of  tech- 
nology from  federal  laboratories  to  commercial  uses.  The  technologies 
represent id  were  varied  in  that  the  cases  studied  ranged  from  the  "Argonne 
Artificial  Kidney”  to  a "synchronous  communications  satellite"  (Teich,  1974: 
1974 :v). 


A conceptual  model  of  technology  transfer  was  formulated  by  the 
researchers.  This  model  encorporated  the  assumption  that  three  basic 
functions  occur  in  the  process  of  technology  transfer: 

Innovation — the  creation  or  the  adaptation  of  the 
technology  . . . j manufacture — the  production  of  the 
technology  for  sale  . . . ; and  utilization— the  ac- 
ceptance and  employment  of  the  technology  (Teich,  1974:2). 

It  was  also  recognized  that  these  three  functions  must  be  brought  together, 
and  a fourth  function  referred  to  as  "brokerage"  was  assumed  to  accomplish 
the  assemblage.  The  ultimate  success  of  technology  transfer  was  hypothe- 
sized to  be  dependent  upon  the  development  of  early  formal  linkages  be- 
tween the  agents  responsible  for  accomplishing  the  above  three  basic  func- 
tions (Teich,  i.?74:2). 

The  Syracuse  group  offered  comments  regarding  key  characteristics 
of  the  technology  transfer  process  and  made  several  observations  regarding 
the  nature  of  the  linkages  that  appear  to  aid  the  transfer  of  technology. 
The  opinions  expressed  by  the  Syracuse  research  group  that  were  used  to 
develop  interview  questions  in  support  of  this  research  effort  are  listed 
below: 

1,  In  our  analysis,  we  were  struck  by  the  fact 
that,  in  each  case,  the  transfer  seemed  to  follow  a 
quite  different  path,  and  that  the  problems  could  not 
be  vi°wed  as  falling  into  neat  categories.  . . . 

...  it  is  questionable  whether  in  any  of  our 
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cases,  the  transfer  could  have  said  to  have  been  planned 
in  detail.  . . . More  aptly,  most  of  those  that  have 
suceeded  seemed  to  have  done  so  by  virtue  of  a set  of 
institutional  arrangements  that  allowed  the  partici- 
pants to  work  out  problems  as  they  come  along  (Teich, 

1974*37,  38). 

2.  In  the  most  successful  cases  the  emphasis  was 
on  solving  a problem  independent  of  a "setting-on-the- 
shelf"  technology.  . . . the  emphasis  was  on  solving  a 
problem,  not  '“inding  a problem  which  can  be  solved  with 
existing  technology  (Teich,  1974*42). 

3.  In  order  for  the  capability-need  matching  to 
take  place,  for  the  dynamic  equilibrium  between  "push'* 
and  "pull"  to  exist,  an  early  informational  linkage 
between  the  innovator  and  the  needer  is  necessary 
(Teich,  1974*43). 

4.  ...  it  appears  that  the  establishment  of  a 
strong  innovator-manufacturer  linkage — strong  enough  sc 
that  the  manufacturer  becomes  a co-innovator — is  one 

of  the  keys  to  successful  transfer  of  federal  labor- 
atory technology  (Teich,  1974*49). 

5.  ...  the  character  of  the  innovator-manufacturer 
linkage  is  strongly  affected  by  the  level  of  maturity  of 
the  technology  in  question.  Maturity  means  not  simply 
the  level  of  development  of  the  technology,  but  the 
degree  to  which  those  involved  in  the  innovation  can 

see  that  the  remaining  technical  and  cost  barriers 
are  soluable  and  are  therefore  willing  to  take  risk 
on  the  innovation  (Teich,  1974*51). 

The  Syracuse  group  also  made  comments  regarding  the  use  of  fed- 
eral laboratories  as  sources  of  technology  for  private  use  and  offered 
some  recommendations  for  improving  technology  transfer.  Some  of  these 
findings  were  considered  in  formulating  additional  technology  transfer 
mechanisms  that  might  be  used  within  the  Air  Force.  The  principal  ones 
considered  are  listed  below* 

1.  . . . because  of  their  very  institutional  nature, 
federal  laboratories  are  less  than  ideal  places  from 
which  to  expect  complete  technology  transfer  to  the 
private  sector.  The  most  basic  problem  is  one  of  in- 
stitutional motivation.  Regardless  of  the  missions  . . . 
the  overriding  goal  is  organizational  survival  (Teich, 

1974*58). 
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2.  If  funding  for  development  of  subsequent  in- 
novations were  somehow  contingent  upon  the  completion  of 
of  technology  transfer , the  innovators  might  be  more 
motivated  to  see  their  innovations  through  to  comple- 
tion, but  there  appears  to  be  little  connection  between 
this  aspect  of  past  performance  and  ability  to  acquire 
new  funds  (Teich,  1974:59). 
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3.  Our  study  suggests  that  the  dominant  mode  of 
existing  institutional  relationship  in  technology 
transfer  relies  too  much  to  the  invisible  hand  of 
mutual  organizational  adjustment.  . . . 

. . . what  are  needed  are  "champions" — or  rather 
"advocates" — of  technology  transfer  as  a process. 

There  are  individuals  that  are  not  pr oduc t-but-proce s s- 
oriented.  They  are  professionals  in  technology  trans- 
fer. . . . They  intervene  at  key  points  in  the  transfer 
process.  . . . They  structure  organizational  relations; 
they  approach  transfer  with  a strategic  plan.  . . . 

They  invest  money  for  necessary  adaptive  engineering. 

. . . They  are  willing  to  risk  failure  (Teich,  1974:62,  64). 

Gartaer  and  Naiman  Effort.  Writing  in  Research  Management, 


i 

* 

I 

i 

t 

j 


Dr.  Gartner  and  Dr.  Naiman  raised  the  issue  that  recent  studies  have  un- 
covered many  harriers  to  the  rapid  and  effective  transfer  of  technology. 
Drawing  upon  their  own  experience  and  other  research  efforts  sponsored 
by  the  National  Science  Foundation  (NAE,  1974;  Michaels,  1973),  the 
authors  formulated  several  institutional  factors  which  appear  to  impede 
the  optimal  rate  of  technology  transfer. 

The  environment  in  which  technology  is  transferred  is  conceived 
of  as  consisting  of  three  systems:  the  General  System,  the  Subsystems, 
and  the  Elements.  The  definition  presented  for  these  three  systems  is 
as  follows: 

1,  The  General  System  is  the  totality  of  a Federal, 
University,  Private  R&D  Laboratory  or  a company. 

2,  The  Subsystems  are  the  department  or  division 
within  a laboratory  or  company. 

3,  The  Elements  are  the  people  in  respective  de- 
partments that  may  or  would  be  directly  involved  in 
technology  transfer  and  utilization  (Gartner,  1976:22). 
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Technology  transfer  efforts  can  normally  be  initiated  at  any  of 
the  above  levels.  However,  the  authors  indicated  that  barriers  to  tech- 
nology transfer  exist  at  each  of  these  levels.  The  specific  barriers 
identified  are  listed  be lows 

Between  the  General  Systems 

1.  No  formal  transfer  policies 

2.  Cost  barriers 

3.  Time  horizon  conflict 

4.  Infringement  problems 

Between  Subsystems 

1.  Inertia  barrier 

2.  Lack  of  an  incentive  structure 

3.  Cost  barrier 

4.  Communication  barrier 

5.  Time  barrier 

6.  Geographic  distance 

7.  Non-existent  transfer  management  structure 

8.  Technology  barrier 

Between  Elements 

1.  Lack  of  an  incentive  structure 

2.  High  risk  of  being  blamed  for  failure 

3.  Insecurity  of  retaining  job  if  not  successful 

4.  Mutual  disrespect 

5.  Unique  requirements  of  each  subsystem 

6.  Updating  of  technology  needs 

7.  Time  barrier 

8.  Lack  of  transfer  organization  managers  (Gartner, 

1976:23) 


Chapter  Summary 

This  chapter  is  an  overview  of  four  separate  concepts  which  are 
related  to  this  research  effort.  The  research  and  development  (R&D)  pro- 
gram of  the  Air  Force  i«>  divided  into  five  categories  of  activity:  research, 
exploratory  development,  advanced  development,  engineering  development,  and 
management  and  supp  >rt.  These  categories  of  activity  are  differentiated 
by  a consideration  of  the  level  of  technical  knowledge  existing  in  the 
field  of  investigation,  the  scope  and  thrust  of  the  activity,  and  the 
means  of  program  control  and  funding.  Each  category  of  activity  is  also 
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subdivided  into  program  elements,  and,  generally,  funds  are  allocated  at 
the  program  element  level. 

The  product  divisions  and  laboratories  are  responsible  for  dif- 
ferent activities.  The  scope  of  the  product  division  efforts  generally 
encompasses  engineering  development  activities;  however,  some  system  re- 
lated advanced  development  activities  are  also  conducted.  The  laboratories 
are  responsible  for  the  development  of  the  technology  base  and  are  required 
to  provide  assistance  in  solving  technical  problems  to  other  Air  Force 
agencies.  The  laboratories  develop  the  technology  base  by  conducting 
L ••search,  exploratory  development,  and  advanced  development  efforts.  Both 
the  laboratories  and  the  product  divisions  rely  heavily  on  industry  con- 
tractors for  the  conduct  of  the  actual  development  work. 

Much  of  the  background  and  the  details  of  the  ADTP  assessment 
process  are  present  elsewhere  in  this  thesis.  The  discussion  presented 
in  this  chapter  is  an  overview  of  the  assessment  process  as  it  applies  to 
new  and  continuing  programs.  This  primary  difference  in  the  application 
of  the  pjrocess  to  these  two  categories  of  ADTPs  is  in  the  details  of  the 
planning  documentation  prepared  by  the  laboratories,  the  required  depth 
of  the  assessments  prepared  by  the  product  divisions,  and  the  timing  of 
the  assessments.  An  overview  of  the  relationship  of  the  ADTP  assessments 
to  the  formulation  of  an  AFSC  advocacy  position  regarding  which  research, 
exploratory  development,  and  advanced  development  programs  should  be  pair- 
sued  by  the  laboratories  is  also  given. 

The  final  topic  presented  in  this  chapter  is  an  overview  of  the 
related  research  on  the  transfer  of  technology.  Several  studies  have  been 
conducted  by  governmental  agencies  and  interested  individuals  on  this 
subject,  and  many  of  these  studies  have  identified  factors  which  app>ear 
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to  either  aid  or  represent  a barrier  to  the  transfer  of  technology.  This 
research  effort  draws  upon  the  results  of  these  prior  studies  in  that  many 
of  the  interview  questions  were  developed  by  considering  the  above  factors. 
Project  Hindsight  conducted  by  the  Department  of  Defense,  a study  conducted 
by  the  Syracuse  University  Research  Corporation,  and  the  recent  work  of 
Dr.  Joseph  Gartner  and  Dr.  Charles  S,  Naiman  are  reviewed.  Chapter  III 
follows,  and  it  is  a discussion  of  the  research  methodology. 
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III.  Research  Methodology 

This  study  deals  with  the  measurement  of  the  perceived  effec- 
tiveness of  the  advanced  development  technology  program  (ADTP)  assess- 
ment process  with  respect  to  alleviating  specific  problem  areas  that  were 
identified  by  a recent  Air  Force  Laboratory  Utilization  Study  (LUS)  effort. 
This  measurement  is  accomplished  by  obtaining  feedback  from  personnel  who 
have  been  involved  in  formulating,  implementing,  and  executing  the  assess- 
ment policy.  The  methodology  used  during  the  course  of  the  research  effort 
consisted  of  four  principal  phases  of  activity:  formulating  a schedule  of 
interview  questions,  scheduling  and  conducting  interviews,  reducing  and 
analyzing  the  data  collected  during  the  interviews,  and  synthesizing  con- 
clusions and  recommendations.  The  process  of  formulating  the  interview 
questions,  the  questions  themselves,  and  the  methodology  for  scheduling 
and  conducting  the  interviews  are  discussed  in  detail  in  this  chapter. 

An  overview  of  the  method;;  used  in  the  reduction  of  the  interview  data  is 
also  provided.  The  analysis  of  the  data  and  the  synthesis  of  conclusions 
and  recommendations  are  presented  in  Chapters  IV  and  V respectively. 

Interview  Questions 

Initial  Considerations . One  of  the  first  items  that  had  to  be 
decided  upon  was  the  mechanism  to  be  used  in  gathering  information  for  the 
research  effort.  The  goal  of  measuring  the  perceived  effectiveness  of  the 
ADTP  assessment  process  required  that  the  views  of  personnel  who  were 
familiar  with  the  assessment  process  be  determined.  This  could  be  ac- 
complished through  either  personal  interviews  or  survey  questionnaires. 
There  are  advantages  and  disadvantages  to  either  approach,  but  ultimately 
a structured  interview  approach  was  chosen. 
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This  structured  interview  technique  was  attractive  because  of  its 
flexibility.  The  structured  design  allows  for  the  collection  of  data  which 
can  be  subjected  to  rigorous  statistical  test,  but,  at  the  same  time,  the 
interview  format  allows  for  verbal  communication  between  the  researcher 
and  those  interviewed.  This  verbal  communication  not  only  allows  for  col- 
lecting subjective  comments  which  can  be  used  to  complement  the  structured 
data,  but  it  also  provides  a means  whereby  ambiguities  in  the  questions 
can  be  resolved.  An  opportunity  to  resolve  ambiguities  in  the  questions 
was  considered  highly  desirable  because  of  the  unique  nature  of  the  ques- 
tions. Also,  it  was  anticipated  that  the  number  of  trial  interviews  con- 
ducted in  order  to  pretest  the  interview  questions  would  be  limited  be- 
cause of  the  time  constraints  placed  on  the  thesis  effort. 

The  most  serious  shortcoming  of  the  structured  interview  approach 
is  that  only  a small  number  of  personnel  can  be  conticted.  The  fact  that 
three  weeks  of  active  interviewing  were  required  in  order  to  complete 
the  thirty-eight  interviews  conducted  for  this  research  effort  is  indic- 
ative of  the  time  required  to  schedule  and  conduct  these  interviews. 

Formulation  and  Review  Process.  The  schedule  of  interview  ques- 
tions was  developed  through  a process  of  question  formulation  and  review. 
The  entire  process  required  approximately  six  weeks  to  complete.  Initially 
the  official  correspondence  which  established  the  assessment  process  and 
the  appropriate  Air  Force  and  Air  Force  Systems  Command  regulations  were 
reviewed  in  detail.  Also,  an  extensive  literature  search  was  conducted 
using  the  research  facilities  of  the  Air  Force  Institute  of  Technology 
(AFIT)  library.  This  literature  search  encompassed  documents  available 
from  the  Defense  Documentation  Center  (DDC),  reports  listed  in  the  Govern- 
ment Reports  Annual  Index  from  1966  to  1975,  and  previous  theses  and 
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professional  studies  listed  in  the  Air  University  Abstracts  of  Research 
Reports.  Several  research  and  development  related  journals  such  as 
Research  Management  were  also  reviewed.  The  information  gathered  during 
this  documentation  review  was  used  to  formulate  a preliminary  schedule  of 
interview  questions. 

This  set  of  preliminary  questions  was  reviewed  and  discussed  with 
the  researcher's  thesis  advisor  and  several  laboratory  and  product  divi- 
sion personnel  who  had  provided  comments  regarding  the  assessment  process 
in  response  to  Gen  Hudson's  29  January  letter.  The  comments  received 
during  this  review  process  provided  the  basis  for  developing  a subsequent 
set  of  interview  questions.  A trip  was  then  made  to  Andrews  Air  Force 
Base,  and  these  questions  were  reviewed  and  discussed  with  HQ  AFSC  personnel 
who  had  participated  in  the  formulation  of  the  assessment  policy.  The 
comments  received  and  the  additional  information  obtained  regarding  the 
background  of  the  assessment  policy  was  then  used  to  further  revise  the 
schedule  of  interview  questions. 

The  third  version  of  the  interview  schedule  was  used  to  conduct 
four  trial  interviews.  Only  minor  modifications  were  made  to  the  ques- 
tions following  these  trial  interviews,  and  the  data  collected  during 
these  interviews  are  included  in  the  data  base  used  for  analysis. 

The  interview  schedule  formulated  as  a result  of  these  activities 
is  shown  in  Appendix  C.  The  nineteen  questions  developed  are  designed 
to  gather  information  in  the  following  areas:  Cl)  demographics,  (2)  effect 
of  ADTP  assessments  on  technology  transfer,  (3)  effect  of  ADTP  assessments 
on  communication  and  dialogue,  (4)  relative  ease  or  difficulty  of  accom- 
plishing principal  assessment  tasks,  (5)  laboratory  responsiveness,  (6)  gen- 
eral attitude  of  personnel  toward  the  assessment  process,  and  (7)  the 
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relative  merit  of  additional  technology  transfer  mechanisms.  The  intent 
of  the  specific  questions  and  many  of  the  considerations  that  had  to  bo 
made  in  developing  these  questions  are  discussed  below. 

Demographics.  The  demographic  factors  used  in  this  research 
effort  are  the  individual's  organization,  grade,  time  of  service  in  the 
research  and  development  field,  and  time  in  present  job  assignment.  Ques- 
tions 1 through  4 were  used  to  collect  this  information.  It  was  felt  that 
the  answers  to  the  interview  questions  would  be  the  most  dependent  upon 
these  factors.  The  individual's  organization  was  deemed  important  because 
of  the  obvious  fact  that  the  work  of  one  organizational  entity  (the  lab- 
oratory) is  assessed  by  another  (the  product  division).  The  individual's 
grade  was  included  because  it  /as  felt  that  there  might  be  a difference 
in  the  answers  given  by  the  higher  grade  personnel  as  opposed  to  those 
given  by  the  lower  grade  personnel.  Similarly  the  length  of  service  in 
the  R&D  field  and  the  time  in  the  present  job  assignment  were  included 
because  it  was  felt  that  the  answers  to  the  interview  questions  might  de- 
pend upon  these  longevity  factors. 

Personnel  working  in  the  product  divisions  were  asked  the  addi- 
tional demographic  question  stated  below* 

Question  5:  If  you  work  in  a product  division,  please 

select  the  one  statement  which  comes  closest  to  repre- 
senting the  degree  to  which  you  have  participated  in  ADTP 
assessments.  I have* 

Seven  different  responses  were  listed,  and  these  responses  ranged  from 
"not  heard  of  the  ADTP  assessment  requirement  before  today"  to  "been  a 
focal  point  for  gathering  comments  for  ADTP  assessment."  Other  personnel 
were  intentionally  not  asked  this  question  because  it  was  recognized  that 
they  would  primarily  be  advanced  development  technology  program  managers. 
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It  was  felt  that  the  degree  of  participation  of  these  individuals  in  the 
assessment  process  would  be  defined  by  virtue  of  their  work  assignments. 

Technology  Transfer.  The  measurement  of  the  perceived  effect  of 
the  ADTP  assessment  process  on  the  transfer  of  technology  was  accomplished 
through  interview  Questions  6,  7,  and  8.  In  these  questions  the  person 
interviewed  was  asked  to  indicate  if  he  felt  that  the  assessment  process 
would  foster  events  which  appear  to  aid  the  transfer  of  technology  and 
if  he  felt  that  the  assessment  process  would  change  the  significance  of 
several  proposed  barriers  to  technology  transfer. 

A seven-point  scale  was  provided  for  the  purpose  of  recording  the 
answers  to  these  questions.  The  available  responses  were  "strongly  dis- 
agree," "disagree,"  "inclined  to  disagree,"  "undecided,"  "inclined  to 
c.gree,"  "agree,"  and  "strongly  agree."  This  seven-point  scale  was  chosen 
over  a more  standard  five-point  scale  in  order  to  reduce  the  number  of 
tied  responses.  It  was  anticipated  that  the  small  size  of  the  data  base 
resulting  from  the  interview  approach  would  dictate  the  use  of  nonpara- 
metric  statistical  tests  in  analyzing  the  interview  results.  Many  of 
these  tests  are  based  upon  ranks,  and  they  generally  require  a minimum 
number  of  ties.  The  specific  tests  used  are  discussed  more  throughly  in 
the  section  on  data  reduction. 

Question  6 was  designed  to  determine  if  the  personnel  interviewed 
perceived  that  the  assessment  process  would  encourage  the  occurrence  of 
certain  events  which  appear  to  aid  the  transfer  of  technology.  The  specific 
events  that  the  personnel  interviewed  were  asked  to  consider  are  listed 
belowt 

1.  Conduct  of  planning  activities  which  serve  to  expose  potential 
problems  before  they  become  serious. 
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2.  The  establishment  of  a forum  within  which  problems  may  be 
resolved  as  they  come  along. 

3.  The  emergence  of  strong  manufacturer  (e.g.,  product  division) 
support  of  a technology  resulting  in  a balance  between  "tech- 
nology push"  and  "demand  pull." 

4.  Establish  that  the  technology  is  sufficiently  rature  so  that 
those  involved  can  see  that  the  remaining  technical  and  cost 
barriers  are  soluble. 

5.  The  existance  of  an  early  informational  linkage  between  the 
innovator  (e.g.,  the  laboratory)  and  the  needer  (e.g.,  the 
product  division  and/or  the  user). 

These  five  aiding  events  were  developed  primarily  from  a consider- 
ation of  the  results  of  the  study  conducted  by  the  Syracuse  University 
Research  Corporation  (see  Chapter  II).  The  use  of  these  events  in  this 
research  effort  was  considered  highly  desirable  because  of  the  similarity 
of  the  "lia'tage"  concept  expressed  by  the  Syracuse  group  and  the  concept 
of  "ties"  between  the  laboratories  and  the  product  division  implied  by 
the  suggestions  made  by  the  Air  Force  LUS  group  (Allen,  1975:26).  However, 
it  was  felt  that  some  corroboration  of  the  applicability  of  the  Syracuse 
group's  results  to  this  research  effort  was  required.  This  seemed  necessary 
because  the  Syracuse  study  addressed  the  transfer  of  technology  from  fed- 
eral laboratories  to  commercial  use  rather  than  from  federal  laboratories 
to  weapon  system  application. 

Two  different  approaches  were  followed  in  order  to  obtain  the  de- 
sired corroboration.  First,  the  management  factors  identified  by  the 
Project  Hindsight  study  team  as  those  which  appear  to  enhance  the  utiliza- 
tion of  technology  were  compared  with  the  linkages  required  for  technology 
transfer  that  were  identified  by  the  Syracuse  study  group.  Although  the 
findings  of  the  two  study  groups  were  not  identical,  it  is  felt  that  a 
great  degree  of  similarity  exists.  For  example,  the  Project  Hindsight 
study  team  found  that  most  utilized  technology  comes  from  programs  in 
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response  to  particular  DOD  problems  (ODDR&E,  1969:40).  The  corresponding 
finding  by  the  Syracuse  study  group  was  that  "in  the  most  successful  cases 
the  emphasis  was  on  solving  a problem  independent  of  a ' setting-on-the- 
shelf'  technology.  ..."  (Teich,  1974:43).  Another  comparison  that  car. 
be  made  is  the  finding  regarding  the  need  for  a demand  for  the  technology 
to  be  established.  The  Project  Hindsight  team  found  that, in  almost  85% 
of  the  utilized  technology  advances  studied  that  tht  application  engineering 
group  originally  expressed  the  need  for  a problem  resolution  (ODDR&E,  1969: 
47).  Correspondingly,  the  Syracuse  group  concluded  that  a capability-need 
matching  was  required — to  establish  a balance  between  technology  push  and 
demand  pull  (Teich,  1974:43).  Other  comparisons  can  be  mare.  The  results 
of  these  two  studies  are  briefly  summarized  in  Chapter  II  for  the  reader's 
convenience. 

The  second  approach  taken  to  corroborate  the  use  of  the  Syracuse 
study  group's  results  was  to  identify  which  Air  Force  agencies  perform  the 
basic  functions  of  innovation,  manufacture,  and  utilization  that  were  iden- 
tified by  the  Syracuse  study  group  (Teich,  1974:2).  The  laboratories,  by 
virtue  of  their  efforts  to  develop  the  technology  base,  are  involved  with 
the  creation  or  adaptation  of  technology.  They  can  be  considered  as  the 
agency  responsible  for  innovation.  The  product  divisions,  by  virtue  of 
their  efforts  to  manage  the  acquisition  of  new  weapon  systems  are  involved 
in  the  production  of  technology  for  use  by  the  operational  commands.  The 
product  divisions,  therefore,  accomplish  a function  similar  to  that  of  the 
me  '.ufacturer.  The  operational  commands  can  be  thought  of  as  the  user  in 
that  they  employ  the  technology  "manufactured"  by  the  product  division. 

Questions  7 and  8 were  designed  to  determine  the  perceived  sig- 
nificance of  several  factors  thought  to  be  barriers  to  technology  transfer, 
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Question  7 addressed  the  situation  perceived  before  the  assessment  process 
wont  into  effect,  and  Question  8 addressed  the  situation  perceived  after 
the  assessment  process  went  into  effect.  The  obvious  intent  of  these  two 
questions  was  to  determine  if  the  ADTP  assessment  process  was  perceived  to 
affect  the  significance  of  any  of  these  barrier  factors. 

The  proposed  barrier  factors  that  the  personnel  were  asked  to  con- 
sider in  answering  Questions  7 and  8 are  listed  in  Table  II.  The  eight 
primary  factors  are  identical  to  the  barriers  identified  as  existing  be- 
tween organizational  subsystems  by  Dr.  Gartner  and  Dr.  Naiman  (Gartner, 
1976:23).  The  subfactors  identified  in  Table  II  were  formulated  by  the 
researcher,  but  they  were  based  upon  the  work  accomplished  by  a National 
Science  Foundation  sponsored  study  designed  to  investigate  barriers  to 
innovation  in  industry  (Michaels,  1973:120-151). 

Gartner  and  Naiman  identified  three  organizational  "systems"  which 
comprise  the  environment  for  technology  transfer  and  identified  a set  of 
barriers  to  this  transfer  for  each  organizational  system  level  (see 
Cnapter  II).  Ideally,  the  perceived  significance  of  the  barriers  at  all 
three  organizational  system  levels  would  have  been  determined.  However, 
this  would  ' ave  required  the  personnel  interviewed  to  consider  twenty  dif- 
ferent barrier  factors,  and  it  was  felt  that  such  a task  would  have  been 
too  time  consuming  for  the  interview  technique.  It  was  therefore  decided 
to  select  the  one  organizational  system  level  that  best  represented  the 
level  at  which  technology  is  transf erred  to  weapon  system  application. 

The  barrier  factors  corresponding  to  the  selected  organizational  system 
level  would  then  be  used  in  Questions  7 and  8. 

The  selection  of  the  subsystem  level  was  based  upon  the  following 
considerations.  The  transfer  of  technology  to  weapon  system  application 
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TABLE  II 

DEFINITION  OF 
PROPOSED  BARRIERS 


1.  INERTIA  BARRIERS 

Product  too  different  from  what  has  been  done  in  the  past. 

A predominant  commitment  to  "current  technology"  rather  than 
supporting  advanced  technology  concepts  and  subsystem. 
Uncertainty  of  internal  predictions  of  cost,  schedule,  and 
performance. 

2.  LACK  OF  AN  INCENTIVE  STRUCTURE 

Advanced  Development  Technology  Program  funding  technique. 
Discouragement  of  long  term  efforts. 

Individual  fear  of  being  blamed  for  failure. 

Lack  of  a laboratory  performance  evaluation  system. 

3.  COST  BARRIERS 

Unavailability  of  cost  information  critical  to  decision  making. 
Predicted  ADTP  cost  too  high. 

No  way  to  evaluate  expected  "return-on-investment"  of  an  ADTP. 
Excessive  cost  required  to  "re-engineer”  an  ADTP  product  before 
it  can  be  used  in  a system  application. 

Lack  of  funds  to  support  an.  ADTP,  including  contract  funding. 

4.  COMMUNICATION  BARRIERS 

Lack  of  coordination  among  various  agencies  or  offices  (e.g., 
staff  with  engineering,  laboratories  with  product  divisions, 
etc.). 

Functional  specialist  have  inadequate  understanding  of  other 
function  (e.g.,  engineering  of  laboratory  or  vice  versa). 
Customer  needs  cannot  be  easily  translated  into  an  ADTP 
definition. 

5.  TIME  BARRIERS 

Unavailability  of  schedule  information  critical  to  decision 
making. 

Lack  of  sufficient  calender  time  to  complete  ADTP, 

Conflict  between  time  horizon  of  product  division  customers 
and  the  laboratories. 

High  risk  of  early  obsolescence. 
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6.  GEOGRAPHIC  DISTANCE 

Distance  between  product  division  and  laboratory. 

Distance  between  ADTP  contractor  and  laboratory. 

7.  NONEXISTENT  TRANSFER  MANAGEMENT  STRUCTURE 

Lack  of  clear  procedures  and  policies  for  approving  and/ 
or  reviewing  an  ADTP. 

Lack  of  a specific  policy  to  insure  transition  of  laboratory 
developed  equipment  to  the  product  divisions. 

Existence  of  a prevalent  "NIH"  syndrome. 

No  mechanism  for  the  transfer  of  technical  knowledge. 

Limited  mobility  of  individuals. 

8.  TECHNOLOGY  BARRIERS 

Technology  base  not  suitable  for  advanced  development. 
Unavailability  of  information  critical  to  predicting  the 
performance  of  an  ADTP  product. 


(Gartner,  1975:23;  Michaels,  1973:120-151) 
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generally  occurs  with  the  transition  from  advanced  development  to  engi- 
neering development,  that  is,  the  transition  from  experimental  hardware 
to  hardware  being  designed  for  service  use  (AFR  80-1,  1970:2).  Also,  the 
agencies  generally  responsible  for  these  two  activities  are  the  AFSC  lab- 
oratories and  the  AFSC  product  divisions.  Thus,  the  transition  generally 
occurs  within  the  organization  responsible  for  managing  Air  Force  R&D — AFSC. 

It  was  felt  that  Dr.  Gartner's  definition  of  the  subsystem  level  was  more 
compatible  with  the  situation  described  above  than  his  definition  of  the 
General  System  level.  The  General  System  level  appears  more  descriptive 
of  the  transfer  which  takes  place  between  the  AFSC  and  other  Air  Force 
commands. 

Communication  and  Dialogue.  Communication  is  interpreted  as  rel- 
ative to  both  verbal  and  nonverbal  forms  of  information  exchange.  Also, 
it  was  postulated  that  the  assessment  process  might  affect  the  overall 
communication  pattern  of  laboratory  and  product  division  personnel.  That 
is,  the  amount  of  time  that  these  personnel  would  spend  in  communication 
with  personnel  working  in  agencies  other  than  the  laboratories  and  product 
divisions  might  also  be  affected.  Thus,  the  interview  questions  pertaining 
to  communication  were  designed  to  address  the  following  three  factors: 

(1)  the  increase,  if  any,  of  the  visibility  of  ADTPs  within  the  labora- 
tories and  product  divisions,  (2)  perceived  changes  to  an  individual's 
pattern  of  communication  with  various  Air  Force  agencies  regarding  new 
ADTPs,  and  (3)  the  adequacy  of  expected  communication  between  laboratory 
and  proouct  division  personnel.  Interview  Questions  9,  10,  11,  12,  and 
13  addressed  these  factors. 

Dialogue  is  interpreted  as  being  primarily  a verbal  exchange  of 
information.  The  anticipation  of  increased  dialogue  between  laboratory 
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and  product  division  personnel  was  measured  by  interview  Question  14. 

Questions  9 and  10  were  designed  as  companion  questionst 

Question  9:  Below  is  a list  of  advanced  development 

programs/projects  either  being  considered  or  currently 
ongoing  in  the  laboratories.  Please  place  a check  mark 
beside  a program/ project  if  you  learned  of  it  because 
of  the  ADTP  assessment  policy/ activity. 

Question  10:  Can  you  think  of  any  other  advanced  devel- 

opment programs/projects  that  you  now  know  about  but 
probably  would  not  have  learned  of  if  the  ADTP  assess- 
ment policy/ activity  had  not  occurred? 

A list  of  fifteen  advanced  development  programs  which  had  been  assessed 
by  the  product  division  prior  to  the  start  of  this  research  effort  was 
provided  in  Question  9.  Question  10  was  included  because  the  list  of 
programs  in  Question  9 was  not  all  inclusive.  The  intent  of  these  two 
questions  was  to  obtain  some  indication  of  the  extent  that  the  assess- 
ment process  will  increase  the  visibility  of  advanced  development  programs. 
The  stimulus  for  asking  these  questions  was  one  of  the  suggestions  made  by 
the  Air  Force  LUS  survey  respondents  (Manley,  1974:2). 

Questions  11  and  12  were  designed  as  "before'*  and  "after"  questions: 

Question  11:  What  is  your  best  estimate  of  the  percent 

of  your  working  time  that  was  spent  communicating  with 
personnel  in  the  agencies  listed  below  regarding  new 
ADTPs  before  the  ADTP  assessment  requirement  went  into 
effect? 

Question  12:  What  is  your  best  estimate  of  the  percent 

of  your  working  time  that  is  or  will  be  spent  communi- 
cating with  personnel  in  the  agencies  listed  below  re- 
garding new  ADTPs  now  that  the  ADTP  assessment  require- 
ment is  in  effect? 

The  agencies  listed  in  these  two  questions  were  "higher  headquarters," 
"product  division,"  "laboratories,"  "Air  Force  Logistics  Command  (AFLC)," 
and  "operational  commands."  The  intent  of  these  two  questions  was  to 
determine  if  the  personnel  interviewed  perceived  that  the  ADTP  assessment 
process  either  has  changed  or  will  change  their  communication  patterns. 
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Question  13  was  stated  as  a "should  be"  question. 

Question  13:  How  much  of  your  working  time  should  be 

spent  communicating  with  personnel  in  the  laboratory 
(product  division)  in  order  for  you  to  stay  aware  of 
status  of  advanced  development  programs  (systems) 
which  are  relevant  to  your  job? 

The  gimer al  intent  of  this  question  was  to  determine  if  the  amount  of 
communication  between  personnel  working  in  the  laboratories  and  product 
divisions  was  perceived  to  be  more  or  less  than  it  should  be.  It  was 
felt  that  this  could  be  accomplished  by  comparing  the  answers  received  in 
response  to  this  question  with  appropriate  answers  received  in  response 
to  Question  12. 

The  final  question  related  to  the  communication  and  dialogue 

area  was  stated  as  follows: 

Question  14:  Do  you  think  that  the  ADTP  assessment 

policy  will  improve  the  dialogue  between  yourself  and 
and  the  product  division  (laboratory)? 

The  intent  of  this  question  was  to  obtain  a direct  measure  of  the  perceived 
effectiveness  of  the  assessment  process  with  respect  to  improving  the  dia- 
logue between  the  product  division  and  laboratory  personnel.  Also,  only 
two  choices  were  provided — "yes”  and  "no."  This  was  done  intentionally 
in  order  to  force  a decision. 

Assessment  Tasks.  The  measurement  of  the  relative  ease  or  dif- 
ficulty of  accomplishing  the  principal  tasks  required  by  the  ADTP  assess- 
ment process  was  done  through  the  use  of  interview  Questions  15  and  16. 

The  first  of  these  questions  was  answered  by  the  laboratory  personnel,  and 
the  second  was  answered  by  product  division  personnel.  The  principal  pur- 
pose of  these  questions  was  to  determine  if  significant  problems  were  en- 
countered in  performing  the  assessment  tasks.  Each  question  contained  two 
parts  as  shown  fallow: 
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Question  15a:  If  you  work  in  a laboratory,  how  dif- 

ficult is  it  Cor  "would  it  be")  for  you  to  obtain  what 
you  feel  is  reliable  information  on  each  of  the  fol- 
lowing items  to  include  in  the  technology  program  plan? 

Question  15b:  Do  you  think  that  the  product  divisions 

have  enough  time  and  manpower  to  devote  to  the  assess- 
ment process? 

Question  16a:  If  you  work  in  a product  division,  how 

difficult  is  it  (or  "would  it  be")  for  you  to  perform 
what  you  feel  is  a reliable  assessment  of  an  ADTP  with 
respect  to  each  of  the  following  items? 

Question  16b:  Do  you  think  that  the  product  divisions 

have  enough  time  and  manpower  to  devote  to  the  assess- 
ment process? 

The  items  that  the  personnel  were  asked  to  consider  were  derived 
from  the  general  outline  of  the  in-depth  assessment  paper  which  is  attached 
to  AFSCR  80-19  (see  Appendix  B).  The  items  that  the  laboratory  personnel 
were  asked  to  consider  were  "payoff,"  "schedules,"  "ADTP  cost  estimates," 
"priority,"  and  "alternate  approaches."  The  items  that  the  product  divi- 
sion personnel  were  asked  to  consider  were  "payoff,"  "schedules,"  "ADTP 
cost  estimates,"  "priority,"  and  "decision  options."  The  laboratory  per- 
sonnel were  asked  to  consider  "alternate  approaches"  rather  than  "decision 
options"  because  the  former  item  was  considered  more  representative  of 
the  information  that  would  be  included  in  the  technology  program  plan. 

The  suggestion  of  changing  the  laboratory  items  list  to  include  "alternate 
approaches"  in  lieu  of  "decision  options"  was  made  to  the  researcher  during 
the  trial  interviews  conducted  with  laboratory  personnel. 

A seven-point  scale  was  provided  for  the  personnel  to  mark  their 
responses  in  order  to  reduce  the  number  of  tied  responses.  The  available 
choices  for  answers  were  "virtually  impossible,"  "very  difficult,"  "some- 
what difficult,"  "neither  difficult  nor  easy,"  "somewhat  easy,"  and  "very 
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The  stimulus  for  including  the  second  part  of  these  questions 
was  obtained  during  the  trial  interviews.  Two  of  the  four  persons  inter- 
viewed expressed  some  concern  as  to  whether  enough  time  and  manpower  was 
available  to  devote  to  the  assessment  process.  It  was  therefore  decided 
to  add  a question  addressing  this  issue.  Two  answers  were  provided— "yes" 


and  "no.' 


Laboratory  Responsiveness.  The  degree  of  perceived  laboratory 


responsiveness  was  addressed  by  interview  Question  17. 

Question  17:  Another  jgoal  of  the  ADTP  assessment  policy 

is  to  increase  the  responsiveness  of  the  laboratories 
to  the  product  divisions  with  respect  to  the  conduct  of 
advanced  development  programs.  Please  indicate  your 
thought  on  the  degree  of  responsiveness  that  should 
exist  and  what  will  exist  now  that  the  ADTP  assessment 
policy  is  in  effect. 

This  question  was  included  because  improving  laboratory  responsiveness 
to  the  product  division  was  one  of  the  primary  objectives  of  the  assess- 
ment process  identified  in  Gen  Hudson's  29  January  1975  letter  (AFSC  (CV) 
letter,  29  Jan  1975: Par  Ij  see  Appendix  B).  The  question  was  formulated 
in  a "what  should  be"  and  a "what  will  be"  format  in  order 'to  determine 
if  the  personnel  interviewed  perceived  that  the  laboratories  will  be  more 
responsive  or  less  responsive  to  the  product  division  than  they  should  be. 

A seven-point  scale  was  provided  in  order  to  reduce  the  number  of 
tied  responses.  The  available  choices  for  answers  were  "nonresponsive," 
"neither  responsive  nor  nonresponsive,"  "to  a small  extent  responsive," 
"somewhat  responsive,"  "for  the  most  part  responsive,"  and  "exceedingly 
responsive." 

Personnel  Attitude.  The  general  atcitude  of  the  personnel  inter- 
viewed toward  the  ADTP  assessment  policy  was  measured  with  the  following 
direct  question: 
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Question  18:  The  ADTP  assessment  policy  is  a good  idea. 

The  seven-point  response  scale  for  expressing  agreement  and  disagreement 
which  has  been  previously  discussed  was  used  for  this  question. 

Additional  Technology  Transfer  Mechanisms.  The  last  question 
asked  during  the  interviews  was  designed  to  determine  if  additional  tech- 
nology transfer  mechanisms  might  be  desirable.  The  personnel  interviewed 
were  provided  a description  of  five  possible  additional  actions  that  might 
be  taken  to  further  improve  the  transfer  of  technology.  They  were  then 
asked  to  indicate  if  they  felt  that  these  proposed  actions  had  enough 
merit  to  warrant  farther  consideration.  The  specific  actions  proposed 
are  listed  below: 

1.  Have  the  product  divisions  annually  rate  the  laboratories 
with  respect  to  generating  transferable  technology. 

2.  Have  the  laboratories  annually  rate  the  product  divisions 
with  respect  to  using  the  technology  that  the  laboratories 
generate. 

3.  Both  1 and  2. 

4.  Somehow  make  funding  for  future  ADTPs  conducted  by  the  lab- 
oratories contingent  upon  the  completion  of  technology  trans- 
fer in  the  past. 

5.  Establish  a technology  transfer  agency  staffed  by  professionals 
in  technology  transfer.  This  agency  would  develop  a strategic 
plan,  structure  organizational  relations,  act  as  a product  ad- 
vocate, and  intervene  at  key  points  (with  funds  if  necessary). 
The  agency  would  also  have  to  be  willing  to  risk  failure  and 
not  be  unduly  criticized  when  a new  technology  program  does 
not  prove  successful. 

These  proposed  actions  were  formulated  by  researcher  based  upon  a review 
of  several  recommendations  made  by  the  Syracuse  study  group  (see  Chapter  II). 
The  seven-point  response  scale  for  expressing  agreement  and  disagreement 
which  has  been  previously  discussed  was  used  for  this  question. 

The  primary  intent  of  the  question  was  to  explore  the  idea  of  in- 
corporating additional  technology  transfer  mechanisms  and  to  solicit  comments 
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regard  ini'  this  area.  It  was  felt  that  If  a preference  was  shown  for  one 
of  those  proposed  actions,  further  study  could  be  devoted  to  establish 
the  specific  procedure. 

Interviews 

Initial  Considerations.  Once  it  was  decided  to  use  structured 
interviews  as  the  data  collection  mechanism  for  this  research  effort,  the 
next  major  task  was  to  determine  exactly  with  which  personnel  to  seek  inter- 
views. However,  two  subtasks  had  to  be  accomplished  prior  to  making  this 
determination.  First  a set  of  criteria  for  selecting  the  personnel  to 
interview  was  needed.  The  ultimate  goal  of  the  research  effort  required 
that  the  laboratory  and  product  division  personnel  who  are  charged  with 
implementing  and  executing  the  process  be  interviewed.  This  implied  that 
the  interviews  be  directed  toward  laboratory  personnel  who  manage  ADTPs 
and  product  division  personnel  who  perform  the  assessments.  It  was  felt 
that  the  supervisors  of  these  personnel  and  key  people  in  the  planning 
offices  should  also  be  considered. 

With  the  above  criteria  in  mind  an  attempt  was  made  to  estimate 
the  size  of  the  population  to  be  sampled.  Although  the  estimates  are 
inexact,  information  obtained  during  the  interview  question  formulation 
review  suggested  that  anywhere  from  100  to  300  personnel  assigned  to  the 
laboratories  and  product  division  under  consideration  would  meet  the  es- 
tablished criteria.  Although  it  was  desirable  to  talk  to  as  many  of  these 
people  as  possible,  it  was  recognized  that  the  time  available  for  actually 
conducting  the  interviews  would  impose  a practical  limitation  of  twenty  to 
fifty  interviews.  Thus,  it  was  decided  to  attempt  to  schedule  interviews 
with  those  personnel  who  had  previously  become  the  most  involved  with  the 
assessment  process. 
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The  second  subtask  that  had  to  be  accomplished  was  to  determine 
the  relative  proportion  of  the  interviews  that  should  be  scheduled  with 
laboratory  and  product  division  personnel.  In  theory,  the  appropriate 
proportions  could  be  estimated  by  determining  how  many  laboratory  and  how 
many  product  division  personnel  were  the  "most  involved"  in  the  assess- 
ment process.  Discussions  with  people  on  this  subject  lead  to  estimates 
that  varied  considerably.  Thus,  the  proportions  which  would  have  resulted 
from  these  estimates  were  not  considered  reliable.  It  was  therefore  de- 
cided, somewhat  arbitrarily,  to  seek  approximately  an  equal  number  of 
interviews  with  laboratory  and  product  division  personnel. 

Personnel  Selection.  A logical  starting  point  for  determining 
who  had  been  the  most  involved  in  the  assessment  process  was  to  identify 
what  assessments  had  been  previously  made  by  the  product  division.  Toward 
this  end,  it  was  determined  that  four  in-depth  assessments  were  either  in 
process  or  had  been  completed  by  the  product  division  and  that  "quick 
response"  prioritization  assessments  had  been  completed.  The  personnel 
who  were  focal  points  for  performing  these  in-depth  assessments  and  the 
corresponding  advanced  development  program  managers  in  the  laboratories 
were  identified  with  the  assistance  of  a point  of  contact  established  in 
the  Development  Plans  Office  of  the  product  division.  Additionally,  sev- 
eral other  product  division  and  laboratory  personnel  who  had  been  closely 
involved  with  the  assessment  process  were  identified.  Approximately 
twenty  key  individuals  were  identified  in  this  initial  effort. 

Contact  was  made  with  as  many  of  these  twenty  individuals  as 
possible,  and  interviews  were  scheduled  with  them.  As  these  interviews 
progressed,  additional  personnel  who  had  either  been  involved  in  the  as- 
sessment process  or  who  were  advanced  development  technology  program  managers 
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were  identified,  and  interviews  were  also  sought  with  these  individuals. 

The  determining  factors  as  to  whether  an  interview  was  scheduled  or  not 
were  the  researcher's  success  in  contacting  the  personnel  to  be  interviewed 
and  their  subsequent  availability.  In  all,  thirty-eight  interviews  were 
held. 

Conduct  of  Interviews.  Interviews  with  the  above  personnel  were 
scheduled  during  the  three-week  period  from  29  April  1976  to  20  May  1976. 

The  interviews  were  conducted  on  an  individual  basis  and  generally  were 
completed  without  interruptions.  Four  of  the  interviews,  three  of  them 
with  the  HQ  AFSC  personnel,  were  not  conducted  personally  by  the  researcher. 
Instead  the  individuals  answered  the  questions  separately  and  returned  the 
completed  interview  schedules  to  the  researcher  at  a later  time.  The 
time  required  for  an  interview  ranged  from  half  an  hour  to  three  hours, 
but  normally  the  interview  period  lasted  approximately  an  hour. 

Although  the  interview  questions  were  structured,  the  atmosphere 
of  the  interviews  was  informal  and  relaxed.  The  typical  procedure  fol- 
lowed was  to  allow  the  individual  to  read  the  questions,  select  his  an- 
swers, and  offer  verbal  and  written  comments  as  desired.  The  degree  of 
dialogue  between  the  researcher  and  the  person  interviewed  varied  from 
interview  to  interview.  Generally  the  dialogue  was  both  open  and  frank. 
Many  of  the  verbal  comments,  as  transcribed  by  the  researcher,  and  the 
written  comments,  directly  quoted,  received  during  the  interviews  are 
listed  in  Appendix  E.  These  comments  are  segregated  by  the  question 
which  provided  the  stimulus  for  the  comment  and  by  the  organization  of 
the  person  interviewed — either  laboratory,  product  division,  or  other. 

Structured  Interview  Results.  The  answers  obtained  to  the  struc- 
tured questions  are  summarized  in  Appendix  D.  The  data  base  is  shown  in 
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two  different  formats.  The  first  format  shows  the  number  of  responses  to 
each  answer  for  each  of  the  structured  questions.  In  addition,  alongside 
each  question  and  response  set,  a histogram  of  the  responses  is  given.  A 
"subjective  feel"  for  the  data  can  be  quickly  realized  by  reviewing  the 
histograms  presented  in  this  format. 

The  second  format  presented  is  similar  to  the  first  except  that 
the  histogram  printout  is  suppressed.  Additionally,  the  responses  are 
shown  by  the  organization  of  those  interviewed.  For  a given  answer  choice 
the  top  figure  represents  the  number  of  personnel  that  selected  that  an- 
swer, and  the  bottom  figure  is  the  corresponding  percentage.  The  per- 
centage is  based  upon  the  appropriate  column  total. 


Data  Reduction 

The  primary  reason  for  formulating  a set  of  structured  interview 
questions  was  to  provide  a mechanism  for  collecting  information  that  could 
be  analyzed  in  an  objective  and  systematic  fashion,  lois  section  is  a dis- 
cussion of  the  various  analytical  techniques  used  in  performing  this  anal- 

* 

ysis  of  the  interview  data.  Emphasis  is  placed  upon  indicating  that  the 
techniques  are  applicable  to  the  interview  data  base  and  the  type  of  con- 
clusions that  can  be  drawn.  No  attempt  is  made  to  present  a detailed  math- 
ematical explanation  of  the  analytical  techniques;  however,  suitable  ref- 
erence is  made  to  the  source  documents  so  that  the  interested  reader  can 
obtain  this  information. 

The  analytical  techniques  used  in  analyzing  the  data  are  those 
generally  thought  of  as  being  either  distribution-free  or  nonparametric. 
These  techniques  are  ideally  suited  for  use  in  the  research  effort  in 
that  no  assumptions  are  necessary  regarding  the  nature  (e.g.,  normal)  of 
the  underlying  population  distribution. 
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Assumptions.  The  principal  assumptions  required  are  that  the 
personnel  interviewed  represent  a random  sample  and  that  their  responses 
are  independent  (Conover,  1971t2,  3,  99,  121,  150,  265,  309,  314).  The 
method  by  which  a sample  is  obtained  from  a given  population  determines 
if  it  is  a random  sample.  The  principal  requirement  is  that  the  selection 
method  insures  that  each  of  the  possible  samples  is  equally  likely  to  be 
obtained  (Conover,  1971 t 62). 

It  is  felt  that  the  "equally  likely"  requirement  is  reasonably 
satisfied  by  the  method  used  to  select  which  laboratory  and  product  divi- 
sion personnel  to  interview.  First  potential  personnel  to  be  interviewed 
were  suggested  by  approximately  ten  different  pceple  acting  independently. 
Secondly,  an  attempt  was  made  to  contact  most  of  these  personnel.  Finally, 
those  personnel  who  were  actually  interviewed  were  selected  by  a set  of 
fortuitous  circumstances  which  included  the  researcher's  success  in  con- 
tacting them  and  their  subsequent  availability  for  the  conduct  of  the 
interviews.  Thus,  it  appears  reasonable  to  assume  that  the  two  samples 
obtained  were  as  equally  likely  as  any  other  samples  of  laboratory  and 
product  division  personnel  that  could  have  been  obtained.  It  is  also 
felt  that  the  answers  to  the  interview  questions  represent  independent 
responses.  Except  for  the  four  interviews  which  were  not  conducted  in 
person  by  the  researcher,  all  interviews  were  conducted  in  private.  Also, 
those  interviewed  did  not  have  access  to  the  interview  questions  prior  to 
the  actual  conduct  of  the  interviews.  It  is  felt  that  these  factors 
eliminated  any  opportunity  for  collaboration  between  individuals. 

Statistical  Hypothesis  Testing.  Most  of  the  analytical  techniques 
used  in  analyzing  the  interview  data  are  those  applicable  to  the  class  of 
statistical  problems  known  as  hypothesis  testing.  Consequently  many  of  the 
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conclusions  reached  as  a result  of  the  research  effort  are  based  upon  the 
rejection  of  a specific  statistical  hypothesis.  A brief  review  of  the 
principal  steps  of  hypothesis  testing  procedure  followed  in  this  research 
effort  is  therefore  given  below* 

1.  The  hypotheses  are  stated  in  terms  of  the  pop- 
ulation. 

2.  A test  statistic  is  selected. 

3.  A rule  is  made,  in  terms  of  possible  values 
of  the  test  statistic,  for  deciding  whether  to  accept 
or  reject  the  hypothesis. 

4.  On  the  basis  of  random  sample  from  the  popu- 
lation, the  test  statistic  is  evaluated,  and  a deci- 
sion is  made  to  accept  or  reject  the  hypothesis  (Con- 
over, 1971*76). 

Statistical  hypotheses  are  stated  in  two  forms:  a null  hypothesis 
and  an  alternate  hypothesis.  The  null  hypothesis  is  denoted  H^,  and  it 
normally  corresponds  to  a statement  about  the  population  that  is  desired 
to  be  shown  false.  For  example,  the  statement  "the  personnel  interviewed 
perceived  that  the  ADTP  assessment  process  did  not  affect  the  significance 
of  the  technology  transfer  communication  barrier"  is  representative  of  a 
null  hypothesis.  The  alternate  hypothesis  is  simply  the  negation  of  the 
null  hypothesis  and  is  denoted 

A test  statistic  is  selected  by  virtue  of  selecting  a spe^fic 
statistical  test.  The  statistical  test  procedures  define  the  method  for 
computing  the  value  of  the  test  statistic  and  the  general  rule  for  deciding 
whether  to  accept  or  reject  the  null  hypothesis.  This  rule  is  based  upon 
some  knowledge  regarding  the  values  that  the  test  statistic  will  take  on 
if  the  null  hypothesis  is  m fact  true.  Generally,  this  knowledge  is 
expressed  in  terms  of  the  probability  distribution  of  the  test  statistic. 

The  results  of  hypothesis  tests  performed  as  part  of  this  research 
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effort  are  presented  in  terms  of  a level  of  significance,  denoted  a.  This 
parameter  is  determined  from  the  computed  value  of  the  test  statistic.  It 
is  the  maximum  probability  of  rejecting  a true  null  hypothesis.  For  the 
purposes  of  this  research,  the  null  hypothesis  is  rejected  if  the  level  of 
significance  is  less  than  or  equal  to  the  critical  value  of  .05. 

Kendall’s  Coefficient  of  Concordance.  This  analytical  technique 
provides  a means  to  measure  agreement  or  concordance  among  several  ob- 
servers regarding  the  ranks  they  assign  to  a set  of  objects.  The  principal 
parameter  of  this  analytical  technique  is  called  the  coefficient  of  concord- 
ance and  is  denoted  W.  This  parameter  represents  the  amount  of  overall 
agreement  of  the  observers,  when  taken  as  a group,  regarding  the  ranks  as- 
signed to  the  objects.  If  there  is  perfect  agreement  among  the  observers 
regarding  the  rankings  then  W = 1,  and  if  there  is  no  agreement,  W = 0. 

The  basis  for  computing  the  coefficient  of  concordance  is  the  algebraic 
sum  of  the  ranks  assigned  to  each  of  the  objects  (Kendall,  1955:94,  95). 

In  theory,  the  coefficient  of  concordance  could  be  used  as  a 
test  statistic  to  evaluate  the  null  hypothesis  that  there  is  no  agreement 
among  the  observers  regarding  the  ranks  assigned  to  the  objects,  but  the 
probability  distribution  of  W is  difficult  to  calculate.  Fortunately  a 
test  statistic  which  is  a function  of  the  coefficient  of  concordance  has 
been  developed,  and  this  statistic  follows  a Snedecor's  F probability  dis- 
tribution (Gibbons,  1971:253-255).  This  test  statistic  is  denoted  F and 
is  used  to  compute  the  level  of  significance  when  testing  a null  hypothesis. 

If  an  observer  assigns  an  equal  rank  to  two  or  more  objects,  then 
these  objects  are  considered  as  being  tied  in  the  rankings.  The  effect  of 
tied  rankings  is  to  reduce  both  the  coefficient  of  concordance  and  the 
F statistic.  A correction  for  tied  rankings  has  been  developed  by  Gibbons 
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(Gibbons,  1971t233,  256-257).  This  correction  requires  that  the  objects 
with  tied  ranks  be  assigned  the  average  rank  that  would  have  been  assigned 
to  these  objects  if  a tie  had  not  occurred.  All  computations  of  the  values 
of  the  coefficient  of  concordance  and  the  F statistic  presented  in  the 
thesis  have  encorporated  the  above  correction  for  tied  ranks. 

It  is  felt  that  the  coefficient  of  concordance  can  be  used  in 
analyzing  the  answers  obtained  for  several  of  the  interview  questions. 

Each  person  interviewed  can  be  considered  an  observer,  and  the  factors 
that  the  person  interviewed  was  asked  to  score  can  be  thought  of  as  the 
objects  to  be  ranked. 

An  example  of  a question  to  which  the  technique  can  be  applied  is 
Question  6.  Each  person  interviewed  was  asked  to  indicate  if  he  felt  that 
the  assessment  process  would  encourage  the  occurrence  of  several  events 
which  appear  to  aid  the  transfer  of  technology.  These  events  can  be  con- 
sidered as  objects  for  the  purpose  of  computing  a coefficient  of  concord- 
ance. Also,  the  seven-point  scale  that  the  individual  used  to  mark  his 
answer  is  structured  such  that  the  extent  of  agreement  or  disagreement  can 
be  determined.  This  answer  scale  provides  a means  of  assigning  ranks  to 
the  various  events. 

Tile  procedure  used  to  assign  a rank  to  an  event  is  summarized  in 
Table  III.  A response  of  "strongly  disagree"  is  considered  the  least  pre- 
ferred, and  a response  of  "strongly  agree"  is  considered  as  the  most  pre- 
ferred in  assigning  the  ranks.  Referring  to  Table  III,  the  event  "estab- 
lish an  early  informational  linkage"  is  ranked  "5,:  because  the  event  was 
scored  the  highest — "strongly  agree."  The  events  "planning  activities  which 
may  expose  problems"  and  "provide  a forum  for  problem  resolution"  were  both 
scored  "inclined  to  agree."  Thus,  both  events  were  assigned  an  equal  rank. 
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TABLE  III 


PROCEDURE  FOR  ASSIGNING 
RANKS  TO  QUESTION  6 EVENTS 


EVENT 


INDIVIDUAL’S 

RESPONSE 


Planning  Activities  Whick 
May  Expose  Problems 


Inclined  to 
Agree 


Provide  a Forum  for 
Problem  Resolution 


Inclined  to 
Agree 


Strong  Product  Division  Support 
Establish  That  Technology  Is  Mature 


Undecided 

Disagree 


Establish  an  Early 
Informational  Linkage 


Strongly 

Agree 
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RANK 

ASSIGNED 


3.5 


3.5 

2 
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The  rank  ”3.5”  is  the  average  rank  that  these  two  events  would  have  been 
assigned  had  they  not  been  scored  the  same  (i.e,,  3.5  = (3  + 4)/2).  Con- 
tinuing the  process,  the  events  "strong  product  division  support"  and 
"establish  that  technology  is  mature”  are  ranked  "2"  and  ”1”  respectively 
because  these  events  were  scored  "undecided”  and  "disagree." 

Appropriate  hypotheses  for  this  question  can  be  stated  as  follows: 

HqS  There  is  no  agreement  among  those  interviewed  regarding  which 
events  the  ADTP  assessment  process  will  encourage. 

H^»  There  tends  to  be  some  agreement  among  those  interviewed  re- 
garding which  events  the  ADTP  assessment  process  will  encourage. 

The  decision  whether  to  reject  or  accept  the  null  hypothesis  is  made  of 

the  basis  of  the  value  of  the  level  of  significance  computed  from  the 

F statistic  discussed  earlier. 

Arguments  similar  to  those  above  can  be  made  to  indicate  that  co- 
efficients of  concordance  can  also  be  used  to  analyze  the  responses  obtained 
for  Questions  7,  8,  11,  12,  15a,  16a,  and  19.  The  results  of  these  analyses 
are  discussed  in  Chapter  IV.  The  specific  values  computed  for  the  key 
parameters  are  tabulated  in  Appendix  F, 

True  Preferential  Order . One  shortcoming  of  the  coefficient  of  con- 
cordance test  is  that  rejection  of  the  null  hypothesis  only  indicates  that 
there  is  some  agreement  among  the  observers  regarding  the  ranking  of  the 
objects.  There  is  no  indication  of  the  true  preferential  order  of  the  ob- 
jects. Gibbons  suggests  that  the  preferential  order  might  be  estimated  by 
assigning  ranks  on  the  basis  of  the  rank  sums  computed  in  order  to  determine 
the  coefficient  of  concordance.  She  further  demonstrates  that  this  technique 
results  in  a best  estimate  for  the  preferential  order,  in  a least-squares 
sense  (Gibbons,  1971:255,  256).  This  technique  is  used  in  analyzing  the 
responses  to  questions  for  which  a coefficient  of  concordance  was  computed. 
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Multiple  Comparisons.  The  preferential  ordering  technique  pro- 
posed by  Gibbons  also  exhibits  a shortcoming.  Although  the  true  ranks  can 
be  estimated,  there  is  no  indication  of  whether  the  difference  in  the  ranks 
assigned  is  statistically  significant.  A hypothesis  test  based  upon  mul- 
tiple comparisons  of  the  rank  sums  computed  in  order  to  determine  the  co- 
efficient of  concordance  overcomes  this  shortcoming  (Hollander,  1973:151-154). 

This  multiple  comparison  test  can  be  used  to  determine  if  the 
rank  assigned  by  the  observers,  acting  as  a group,  to  one  object  differs 
statistically  from  the  rank  assigned  to  a second  object.  The  assumptions 
are  the  same  as  those  required  by  the  coefficient  of  concordance  test. 

The  test  statistic  is  computed  as  the  absolute  value  of  the  difference 
in  the  rank  sums  of  the  two  objects.  This  test  statistic  does  not  follow 
a standard  probability  distribution}  however,  tables  which  can  be  used  to 
determine  the  level  of  significance  are  available  (Hollander,  1973:330, 
373-378). 

The  multiple  comparison  test  may  be  applied  to  the  same  questions 
as  the  coefficient  of  concordance  test  because  the  assumptions  are  the 
same.  An  example  of  the  type  of  hypotheses  that  can  be  tested  with  the 
multiple  comparison  test  can  be  formulated  using  Question  6 as  follows: 

H^:  Those  personnel  interviewed  perceived  that  the  occurrence 

of  the  events  "establishment  of  an  early  informational 
linkage"  and  "establish  technology  is  mature"  will  be 
equally  encouraged  by  the  ADTP  assessment  process. 

H^:  Those  personnel  interviewed  perceived  that  the  occurrence 

of  these  two  events  will  not  be  equally  encouraged  by  the 
ADTP  assessment  process. 

Sign  Test.  The  sign  test  is  useful  in  drawing  inferences  about 
data  which  comes  in  x-y  pairs.  The  additional  assumptions  required  by 
this  test,  are  that  there  is  some  natural  basis  for  pairing  the  responses 
and  that  t.ie  measurement  scale  used  to  record  the  responses  is  such  that 
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a pair  can  be  classified  as  a plus  if  x is  less  than  y or  as  a minus  if  x 
is  greater  than  y (Conover,  1971:121). 

This  test  can  be  used  to  test  hypotheses  about  the  responses  to 
Questions  7 and  8,  11  and  12,  12  and  13,  and  17.  The  responses  of  each 
individual  can  be  considered  one  x-y  pair.  The  structure  of  these  ques- 
tions allows  for  natural  grouping  of  the  data  in  terms  such  as  "before" 
and  "after,"  and  "what  should  be"  and  "what  will  be."  Also,  the  measure- 
ment scale  used  in  these  questions  allows  for  the  pluses  and  minuses  to 
be  determined. 

The  test  statistic  for  this  test  is  the  total  number  of  plus 
pairs  obtained,  and  the  probability  distribution  of  the  test  statistic  is 
the  standard  binomial  distribution  (Tables,  1955).  The  sign  test  is  a 
very  popular  nonparametric  test  and  is  discussed  in  detail  in  most  non- 
parametric  statistics  textbooks  (Conover,  1971:121-126;  Gibbons,  1971: 
100-106;  Hollander,  1973:39-45). 

One  situation  encountered  in  using  the  sign  test  to  analyze  the 
interview  data. was  that  many  individuals  marked  the  same  answer  for  the 
x-response  and  the  y-response.  These  tied  observations  result  in  neither 
a plus  or  a minus.  Tied  observations  are  normally  discarded  when  using 
the  sign  test,  but  it  is  felt  that  this  procedure  leads  to  misleading 
conclusions  if  there  are  many  ties.  A conservative  approach  was  there- 
fore adopted  in  order  to  include  the  tied  observations.  This  approach 
is  identified  by  Gibbons  and  requires  that  the  tied  observations  be  con- 
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sidered  as  minuses  when  evaluating  the  level  of  significance  (Gibbons, 
1971:102). 


The  sign  test  is  used  to  test  hypotheses  similar  to  those  listed 


below: 
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H.»  The  personnel  interviewed  perceived  that  the  communication 
barrier  to  technology  transfer  was  less  significant  before 
the  ADTP  assessment  process  went  into  effect. 

H_:  The  personnel  interviewed  perceived  that  the  communication 

barrier  to  technology  transfer  was  more  significant  before 
the  ADTP  assessment  process  went  into  effect. 

Two- sample  Tests.  These  tests  are  useful  in  making  inferences 
regarding  differences  in  the  responses  given  by  two  different  groups  of 
personnel.  Two  different  two-sample  tests  are  used  to  analyze  the  inter- 
view data  collected  during  the  research  effort:  the  Smirnov  test  and  the 
contingency  table  test.  The  distribution  function  for  the  Smirnov  test 
statistic  has  been  extensively  tabled  (Birnbaum,  1960:710-720;  Massey, 
1952:435-441;  Hollander,  1973:397-418),  The  distribution  function  for 
the  contingency  table  test  statistic  is  the  standard  Chi-square  distri- 
bution (Conover,  1971:152).  These  tests  are  in  common  use  and  are  dis- 
cussed in  detail  in  most  nonparametric  statistics  textbooks  (Conover, 
1971:309-314,  149-166).  A detailed  discussion  of  these  tests  is  there- 
fore not  provided. 

Two  different  tests  were  required  because  of  the  structure  of 
the  answer  scale.  The  answer  scale  must  be  such  that  the  responses  can 
be  ranked  from  low  to  high  in  order  to  use  the  Smirnov  test  (Conover, 
1971:309).  This  requirement  is  satisfied  by  the  answer  scales  associated 
with  Questions  6,  7,  8,  11,  12,  13,  15a,  16a,  17,  18,  and  19.  However, 
the  answer  scales  associated  with  Questions  14,  15b,  and  16b  are  simply 
"yes'*  and  "no,"  and  therefore  do  not  satisfy  the  ranking  requirement. 

The  contingency  table  test  was  used  to  analyze  the  responses  to  these 
questions, 

A typical  set  of  hypotheses  which  can  be  tested  with  the  two- 
sample  tests  are  listed  below: 
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HqS  There  tends  to  be  no  difference  between  the  responses  given 
by  the  laboratory  personnel  and  those  given  by  product  divi- 
sion personnel  in  answer  to  Question  18  which  asks  if  "the 
ADTP  assessment  policy  is  a good  idea." 

H^j  Personnel  in  these  two  organizations  tend  to  give  different 
responses  in  answer  to  Question  18. 

Confidence  Interval  Estimation.  This  analytical  technique  is  not 
used  to  test  a statistical  hypothesis.  Instead,  it  is  useful  in  making 
inferences  regarding  some  unknown  parameters  associated  with  a sample  pop- 
ulation. The  confidence  interval  is  composed  of  two  components:  a confi- 
dence coefficient  and  an  interval  estimate  (Conover,  1971:99-103).  The 
confidence  coefficient  must  be  selected  prior  to  using  the  technique.  A 
value  of  95%  has  been  selected  for  this  research  effort.  The  interval 
estimate  is  determined  by  appropriate  computations  and  is  typically  spec- 
ified in  terms  of  a lower  and  an  upper  bounds  for  the  unknown  parameter. 

It  is  felt  that  one  population  parameter  which  is  useful  in  in- 
terpreting the  data  collected  during  this  research  effort  is  the  propor- 
tion of  personnel  who  have  given  a particular  response  to  an  interview 
question.  Confidence  intervals  for  this  parameter  may  be  determined  for 
any  of  the  interview  questions.  However,  they  are  generally  only  computed 
for  those  questions  which  have  a single  set  of  possible  answers,  (e.g.. 
Questions  14,  15b,  16b,  and  18). 


Chapter  Summary 

This  chapter  is  a discussion  of  the  process  used  to  formulate 
specific  interview  questions,  the  questions  themselves,  and  the  methodology 
followed  in  scheduling  and  conducting  interviews  with  personnel  familiar 
with  the  ADTP  assessment  policy.  An  overview  of  many  of  the  analytical 
techniques  used  in  the  reduction  of  the  interview  data  is  also  presented. 

The  interview  questions  were  developed  through  a process  of 
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question  formulation  and  review.  Information  obtained  through  an  extensive 
search  of  relevant  literature  and  comments  received  from  key  personnel  as- 
signed to  HQ  AFSC,  one  product  division,  and  two  laboratories  were  used  to 
formulate  a final  set  of  interview  questions.  The  nineteen  questions  devel- 
oped through  this  process  are  designed  to  gather  information  in  the  following 
areasi  (1)  demographics,  (2)  effect  of  ADTP  assessments  on  technology  trans- 
fer, (3)  effect  of  ADTP  assessments  in  communication  and  dialogue,  (4)  rela- 
tive ease  or  difficulty  of  accomplishing  principal  assessment  tasks,  (5)  lab- 
oratory responsiveness,  (6)  general  attitude  of  personnel  toward  the  assess- 
ment process,  and  (7)  the  relative  merit  of  additional  technology  transfer 
mechanisms.  The  intent  of  the  specific  questions  and  many  of  the  consider- 
ations that  had  to  be  made  in  developing  these  questions  are  discussed  in  de- 
tail in  this  chapter.  The  schedule  of  interview  questions  is  given  in  Appendix  C, 

Only  those  personnel  most  involved  with  the  assessment  process  were 
interviewed.  Primarily  those  interviewed  were  laboratory  personnel  who  man- 
age ADTPs  and  product  division  personnel  who  had  participated  in  the  assess- 
ment process.  However,  the  supervisors  of  some  of  these  personnel  and  a few 
personnel  from  laboratory  and  product  division  planning  offices  were  also 
interviewed.  In  all,  thirty-eight  interviews  were  conducted  during  a three- 
week  period  from  29  April  19/6  to  20  May  1976,  The  data  collected  during 
these  interviews  is  summarized  in  Appendix  D. 

The  analytical  techniques  used  in  analyzing  the  interview  data  are 
those  generally  thought  of  as  being  either  distribution-free  or  nonpara- 
metric,  Most  of  these  techniques  are  applicable  to  the  class  of  statis- 
tical problems  known  as  hypothesis  testing,  but  a method  of  estimating 
confidence  intervals  is  also  discussed.  Chapter  IV  follows  and  is  a de- 
tailed discussion  of  the  actual  data  analysis. 
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IV.  Data  Analysis 

This  chapter  is  a discussion  of  the  analysis  of  the  information 
received  in  response  to  the  questions  asked  during  the  interviews.  The 
formulation  of  the  interview  questions  is  discussed  in  detail  in  Chap- 
ter III,  A summary  of  the  data  collected  during  the  interviews  is  pre- 
sented in  Appendix  D in  two  different  formats.  An  explanation  of  these 
formats  is  presented  in  Chapter  III  under  the  section  entitled  "Struc- 
tured Interview  Results." 

Analysis  Approach 

The  structured  interview  questions  provide  both  the  data  and  the 
framework  for  the  analysis.  These  questions  were  designed  to  gather  in- 
formation in  seven  different  areas  of  interest  (see  Chapter  III),  and 
the  analysis  was  conducted  to  address  each  of  these  areas  separately. 
Within  each  of  the  seven  principal  areas  of  interest  the  analysis  was 
generally  conducted  on  a question  by  question  basis.  The  analysis  of 
the  demographic  questions  being  the  exception. 

Many  factors  were  considered  when  performing  the  question  by 
question  analysis,  and  the  specific  approach  taken  varies  from  question 
to  question.  In  general,  both  the  subjective  comments  received  during 
the  conduct  of  the  interviews  and  the  conclusions  obtained  from  applying 
appropriate  analytical  techniques  were  considered  in  developing  findings. 
The  analytical  techniques  used  are  discussed  in  Chapter  III  under  the 
heading  "Data  Reduction." 

The  use  of  quantitative  methods  in  analysing  the  data  required 
that  each  of  the  descriptive  word  answers  be  converted  to  a numerical 
value.  The  convention  followed  in  assigning  numerical  values  is  shown  in 
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Table  IV,  The  entries  in  the  first  three  table  columns  are  the  descriptive 
word  answers  used  for  the  questions  defined  by  the  table  column  heading. 

The  entries  in  the  last  column  are  the  numerical  value  assigned  to  the 
word  answer  defined  by  the  table  row. 

Demographics 

The  demographic  variables  used  in  this  research  effort  are  the 
individual's  organization,  grade,  time  of  service  in  the  R&D  field,  and 
time  in  present  job  assignment.  The  answers  to  the  questions  used  to 
collect  this  information  provided  the  basis  for  developing  the  specific 
demographic  groupings  defined  in  Table  V.  Note  that  only  the  All  Per- 
sonnel grouping  includes  the  responses  received  from  personnel  not  as- 
signed to  either  a product  division  or  a laboratory. 

The  demographic  groupings  defined  in  Table  V can  be  categorized 
as  either  organizational  or  nonorganizational.  The  number  of  people  in- 
terviewed in  each  of  the  organizational  groupings  is  shown  in  Table  VI. 

The  relationship  between  the  organizational  and  nonorganizational  groupings 
is  shown  in  Table  VII.  The  Table  VII  entries  are  the  number  of  personnel 
assigned  to  the  organizational  grouping  defined  by  the  table  row  and  also 
included  in  the  nonorganizational  grouping  defined  by  the  table  column. 

Personnel  working  in  three  different  laboratories  were  interviewed. 
Thirteen  (76%)  of  these  personnel  either  were  or  still  are  managers  of 
ADTPs,  and  the  remaining  four  (24%)  were  assigned  to  the  laboratory  plans 
office.  Almost  half  (47%)  of  the  laboratory  personnel  interviewed  had 
been  working  in  the  R&D  field  more  than  twenty  years,  and  eight  (47%)  had 
been  in  their  present  work  assignments  more  than  two  years.  Also,  five 
(29%)  of  the  personnel  held  grades  and  ranks  that  placed  them  in  the 
higher  grade  demographic  grouping  defined  in  Table  V. 
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TABLE  V 

DEFINITION  OF  DEMOGRAPHIC 
GROUPINGS 


Demographic  Grouping 
and  Short  Title 


Definition 


All  Personnel 
(All) 

All  personnel  interviewed 

All  Laboratories 
(Lab) 

All  laboratory  personnel 
interviewed. 

Laboratory  A Only 
(Lab  A) 

Laboratory  personnel  who  worked 
in  laboratory  A. 

Laboratory  B Only 
(Lab  B) 

Laboratory  personnel  who  worked 
in  laboratory  B. 

Laboratory  C Only  * 
(Lab  C) 

Laboratory  personnel  who  worked 
in  laboratory  C. 

All  Product  Divi- 
sion organizations 
(P.D.) 

All  product  division  personnel 
interviewed. 

Product  Division 
Organization  A Only 
(P.D.  A) 

Product  division  personnel  who 
worked  in  development  planning 
organizations. 

Product  Division 
Organization  B Only 
(P.D.  B) 

Product  division  personnel  who 
worked  in  system  engineering 
organizations. 

GS-15/Lt  Col  and  Abo  ? 
(GS  15  LC  & Above) 

All  laboratory  and  product  divi- 
sion personnel  who  held  the  ci- 
vilian grade  of  GS-15  or  higher 
and  all  military  personnel  who 
held  the  rank  of  Lieutenant 
Colonel  or  higher. 

GS-14/Maj  and  Below 
(GS  14  Maj  & Below) 

All  laboratory  and  product  divi- 
sion personnel  who  held  the  ci- 
vilian grade  of  GS-14  or  lower 
and  all  military  personnel  who 
held  the  rank  of  Major  or  below. 
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TABLE  V CONTINUED 


Demographic  Grouping 
and  Short  Title 


Over  Twenty  Years  Exper- 
ience in  R&D  Field 
(Over  20  Yerrs) 


Under  Twenty  Years  Expel 
ience  in  R&D  Field 
(Under  20  Years) 


More  Than  Two  Years  in 
Present  Job  Assignment 
(More  Than  2 Yrs) 


Less  Than  Two  Years  in 
Present  Job  Assignment 
(Less  Than  2 Yrs) 


Definition 


All  laboratory  and  product  divi- 
sion personnel  who  indicated 
that  they  had  more  than  twenty 
years  experience  in  answer  to 
Question  3. 

All  labo'-vnory  vd  product  divi- 
sion per so; m el  who  indicated 
that  they  had  twenty  years  or 
less  experience  in  answer  to 
Question  3. 

All  laboratory  and  prod,  ct  divi- 
sion personnel  who  indicated 
that  th«  y had  been  in  their 
present  work  assignment  more 
than  two  years  in  answer  to 
Question  4. 

All  laboratory  and  product  divi- 
sion personnel  who  indicated 
that  they  had  been  in  their 
present  work  assignment  less 
than  two  years  in  answer  to 
Question  4. 


Not  Considered  as  a Separate  Demographic  Grouping  in  Statistical 
Analysis  Because  Only  Two  Personnel  Were  Interviewed. 
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TABLE  VI 

NUMBER  OF  PERSONNEL  INTERVIEWED 
BY  ORGANIZATIONAL  GROUPING 


Laboratories  Product  Division 


Other 


Laboratory  A 

6 

Organizational 
Grouping  A 

HQ  AFSC 

3 

Laboratory  B 

9 

(Development 

Operational 

Planning) 

8 

Command  Liaison 

Laboratory  C 

2 

Office 

1 

Organizational 

Total 

Grouping  B 

Total  Other 

A 

Laboratory 

17 

(System 

Engineering) 

9 

Total  Product 
Division 

17 

Total  Interviews:  38 


TABLE  VII 


RELATIONSHIP  BETWEEN  ORGANIZATIONAL  AND 
NONORGANI ZA7T ONAL  DEMOGRAPHIC  GROUPINGS 


GS  15 

GS  14 

Over 

Under 

More 

Less 

LC  & 

Maj  & 

20 

20 

Than 

Than 

Above 

Below 

Years 

Years 

2 Yrs 

2 Yrs 

Lab 

5 

12 

8 

9 

8 

9 

Lab  A 

2 

4 

4 

2 

5 

1 

Lab  B 

3 

6 

4 

5 

2 

7 

Lab  C 

0 

2 

0 

2 

1 

1 

P.D. 

8 

9 

11 

6 

13 

4 

P.D.  A 

2 

6 

5 

3 

8 

0 

P.D.  B 

6 

3 

6 

3 

7 

2 

71 


GSM/SM/76S-21 


The  product  division  personnel  interviewed  were  assigned  to 
organizations  in  two  principal  categories.  These  categories  were  de- 
velopment planning  and  system  engineering  offices.  Eleven  (65%)  of  those 
interviewed  hzd  been  focal  points  for  ADTP  assessments — either  in-depth 
or  "quick  response"  prioritization.  Moreover,  eleven  (65%)  of  the  product 
division  personnel  had  been  working  in  the  R&D  field  more  than  twenty 
years,  and  thirteen  (76%)  had  been  in  their  present  work  assignment  more 
than  two  years.  Also,  eight  (47%)  of  these  personnel  held  grades  and 
ranks  that  placed  them  in  the  higher  grade  demographic  grouping  defined 
in  Table  V. 

Although  the  primary  thrust  of  the  interview  effort  was  directed 
toward  obtaining  feedback  from  laboratory  and  product  division  personnel, 
the  views  of  four  personnel  assigned  to  other  organizations  were  obtained 
as  shown  in  Table  VI.  The  information  obtained  from  these  individuals 
was  used  by  the  researcher  in  order  to  achieve  a broader  perspective  of 
the  assessment  process,  but  generally  the  data  was  not  used  in  the  statis- 
tical test  computations. 

Two  principal  populations  were  sampled  duri.  , the  interview  period. 
One  of  these  was  all  laboratory  personnel  who  were  either  advanced  devel- 
opment program  managers  or  assigned  to  the  laboratory  plans  office  and 
who  were  perceived  by  their  contemporaries  to  be  knowledgeable  of  the 
assessment  process.  The  other  population  sampled  was  all  product  division 
personnel  assigned  to  offices  responsible  for  either  development  planning 
or  systems  engineering  and  who  had  participated  in  the  assessment  process. 
Those  personnel  identified  in  the  Other  organization  category  are  not  con- 
sidered to  be  representative  of  a principal  population  because  the  number 
of  interviews  is  so  small. 
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Technology  Transfer 

The  measurement  of  the  perceived  effect  of  the  ADTP  assessment  pro- 
cess on  the  transfer  of  technology  was  accomplished  through  interview  Ques- 
tions 6,  7,  and  8.  In  Question  6 the  person  interviewed  was  asked  if  he 
felt  that  the  ADTP  assessment  process  would  encourage  the  occurrence  of  cer- 
tain events  which  appear  to  aid  the  transfer  of  technology.  In  Questions 
7 and  8 those  interviewed  were  asked  if  they  perceived  certain  factors  to 
be  a barrier  to  the  transfer  of  technology.  The  conditions  existing  before 
and  after  the  ADTP  assessment  policy  went  into  effect  were  considered. 

Technology  Transfer  Aiding  Events  (Question  6) 

General.  The  specific  events  that  the  personnel  interviewed 
were  asked  to  consider  were  identified  in  Question  6 (see  Appendix  C and 
Chapter  III).  The  mean  response  given  by  the  various  demographic  groups 
defined  in  Table  V for  the  proposed  aiding  events  varied  from  4.00  to 
6.50.  This  numerical  range  corresponds  to  a word  answer  range  from  "un- 
decided" to  "agree."  It  is  felt  that  the  fact  that  these  mean  responses 
tend  toward  the  "agree"  end  of  the  answer  scale  suggests  that  those  inter- 
viewed perceived  chat,  in  general,  these  events  would  be  encouraged  by 
the  assessment  process.  However,  the  analysis  performed  shows  that  some 
of  the  events  are  perceived  to  be  more  likely  encouraged  than  others. 
Perception  of  Events 

Agreement.  Overall  there  tends  to  be  agreement  among 
those  interviewed  regarding  which  events  the  ADTP  assessment  process  will 
most  likely  encourage.  Moreover,  there  generally  tends  to  be  agreement 
among  the  personnel  within  each  of  the  demographic  groupings  considered, 
These  findings  result  from  testing  the  statistical  hypothesis  that  there 
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is  no  agreement  among  those  interviewed  regarding  the  ranking  of  the  propored 
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technology  transfer  aiding  events.  The  coefficient  of  concordance  and  the 
corresponding  level  of  significance  were  computed  from  the  data  representing 
each  of  the  thirteen  demographic  groupings  defined  in  Table  V.  The  specific 
hypothesis  tested  and  the  results  of  the  analysis  are  shown  in  Figure  5. 

The  values  of  the  key  parameters  computed  in  performing  this  test  are  tab- 
ulated in  Appendix  F. 

The  chart  shown  in  Figure  5 is  constructed  to  illustrate  the  value 
of  the  level  of  significance  ($,)  computed  for  each  of  the  demographic 
groupings.  A reverse-logarithmic  scale  is  used  for  plotting  the  level  of 

—5 

significance.  Very  small  values  (<3.0  ) of  a are  plotted  to  the  right  of 

the  scale,  and  large  values  (ft  « 1)  are  plotted  to  the  left  of  the  scale. 
Thus,  if  the  level  of  significance  is  small,  the  bar  will  extend  to  the 
right.  If  the  bar  crosses  the  double  vertical  line,  drawn  at  the  critical 
value  of  .05,  the  null  hypothesis  of  no  agreement  among  the  personnel  is 
rejected.  There  tends  to  be  more  agreement  among  the  personnel  within  a 
specific  demographic  grouping  if  the  bar  extends  far  to  the  right.  The 
concept  of  the  level  of  significance  was  discussed  in  Chapter  III  under 
the  section  entitled  "Statistical  Hypothesis  Testing." 

The  results  shown  in  Figure  5 indicate  that  the  personnel  assigned 
to  laboratory  A and  to  product  division  organization  A did  not  tend  to  ex- 
hibit agreement  regarding  the  ranking  of  the  five  technology  transfer 
aiding  events.  From  Table  F-I  it  can  be  see  hat  the  mean  of  the  ranks 
assigned  to  these  events  by  laboratory  A personnel  are  4.08,  3,08,  2.75, 
2.58,  and  2.50.  Note  that  the  mean  ranks  of  the  fox:c  lowest  ranked  events 
differ  by  only  .58.  A corresponding  value  of  2.00  was  obtained  when  the 
same  computations  were  performed  using  the  mean  of  the  ranks  assigned  by 
personnel  (laboratory  B)  who  did  exhibit  significant  agreement  regarding 
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DEMOGRAPHIC  GROUPING 

All  Personnel 
n = 38 

All  Laboratories 
n = 17 

Product  Division  (All 
Organizations)  n = 17 

Laboratory  A Only 
n = 6 

Laboratory  B Only 
n = 9 

Product  Division  (Org  A 
Only)  n = 8 

Product  Division  (Org  B 
Only)  n *-  9 

GS-15/Lt  Col  and  Above 
n = 13 

GS-14/Maj  and  Below 
n = 21 

Over  20  Yrs  Experience  in 
R&D  Field  n = 19 

Under  20  Yrs  Experience  in 
R&D  Field  n = 15 

More  Than  2 Yrs  in  Present 
Job  Assignment  n = 21 

Less  Than  2 Yrs  in  Present 
Job  Assignment  n - 13 


|-  SIGNIFICANT  AGREEMENT 

T 


1.0  .1  .01  .001  10  ” <10 

LEVEL  OF  SIGNIFICANCE  ~ $. 
(CRITICAL  VALUE  .05) 

HqJ  There  is  no  agreement  among  those  interviewed  regarding  which 
events  the  ADTP  assessment  process  will  encourage, 

H^»  There  tends  to  be  agreement  among  those  interviewed  regarding 
which  events  the  ADTP  assessment  process  will  encourage. 
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Figure  5.  Results  of  Coefficient  ■'f  Concordance  Tests — 
Technology  Trans! or  Aiding  Events  (Question  6) 
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the  ranking  of  the  events.  A comparison  similar  to  the  above  was  made  for 
the  means  of  the  ranks  assigned  by  the  product  division  organization  A per- 
sonnel. For  these  personnel,  the  mean  ranks  of  the  four  lowest  ranked 
events  differ  by  1.12.  Thus,  the  lack  of  agreement  appears  to  be  due  to 
the  ranking  of  the  four  lowest  ranked  events. 

Observation  and  Comment.  It  is  interesting  to  note 
that  the  two  groups  of  personnel  which  failed  to  agree  on  the  ranking  of 
the  aiding  events  were  those  most  involved  with  the  in-depth  assessment 
process.  Three  of  the  four  ADTPs  for  which  in-depth  assessments  held  been 
performed  by  the  product  division  were  laboratory  A managed  programs. 

Also,  the  product  division  organization  A designation  corresponds  to  the 
development  pieinning  office.  This  office  was  the  focal  point  for  conducting 
the  assessment  process  (AFSCR  80-19,  1976»Par  3),  The  hypothesis  that  close 
involvement  with  the  assessment  process  somehow  contributes  to  lack  of 
agreement  regarding  the  ranking  of  the  transfer  aiding  events  is  there- 
fore suggested.  However,  this  hypothesis  cannot  be  tested  with  the  present 
data  base,  and  any  conclusion  drawn  regarding  its  validity  would  be  pure 
speculation.  This  hypothesis  should  therefore  be  investigated  in  any 
future  research  performed  regarding  the  effect  of  the  ADTP  assessment  pro- 
cess on  the  transfer  of  technology. 

Preferred  Order.  Estimates  of  the  true  preferential 
order  of  the  five  technology  aiding  events  were  made  by  assigning  ranks 
to  the  events  as  outlined  in  Chapter  III  (see  Table  III).  Each  demographic 
grouping  was  considered,  and  ranks  were  assigned  if  agreement  among  the  per- 
sonnel was  shown  to  exist  in  Figure  5.  If  the  agreement  was  not  found, 
then  it  was  assumed  that  the  events  were  equally  preferred.  The  ranks 
assigned  to  the  events  are  shown  in  Table  VIII, 
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Table  VIII  is  constructed  such  that  each  aiding  event  is  repre-  ! 

vJ  l 

sented  by  a table  row  and  each  demographic  grouping  is  represented  by  a 'i 

I 

table  column.  The  numerical  values  shown  in  the  table  correspond  to  the 
rank  assigned  to  a specific  aiding  event  by  the  demographic  group  identified 
at  the  top  of  the  table  column.  A value  of  "5’*  indicates  the  most  preferred 
event,  and  a value  of  "1"  indicates  the  least  preferred  event.  The  dif- 
ference between  the  ranks  assigned  to  the  highest  ranking  event  and  the 
ranks  marked  with  a double  asterisk  (**)  was  found  to  be  statictically  sig- 
nificant (&  < .05).  The  ranks  were  marked  with  a single  asterisk  (*)  if 
this  difference  was  found  to  be  only  marginally  significant  (.05  < a < .1). 

Thus,  the  results  shown  in  Table  VIII  indicate  the  estimated  preferential 
order  of  the  proposed  technology  transfer  aiding  events.  However,  it 
should  be  noted  that  his  estimated  order  could  reasonably  be  expected  to 
( ’ change  if  another  series  of  interviews  was  to  be  conducted  unless  the  dif- 

ference between  the  ranks  is  shown  to  be  statistically  significant. 

The  results  shown  in  Table  VIII  also  indicate  which  of  the  ranks  as- 
signed to  the  events  differs  statistically  from  that  assigned  to  the  highest 
ranked  event.  It  can  be  seen  that  this  difference  for  the  lowest  ranked 

event  is  statistically  significant  for  eight  of  the  demographic  groupings, 

♦ 

and  the  difference  is  marginally  significant  for  two  of  the  demographic 
groupings.  The  level  of  significance  increases  when  the  differences  between 
the  rank  assigned  to  the  highest  ranked  event  and  the  rank  assigned  to  the  sec- 
ond lowest  ranked  event  are  tested.  For  this  case  two  of  the  differences  are 
statistically  significant,  and  two  are  marginally  significant.  Only  two  demo- 
graphic groups  ranked  the  events  so  that  the  differences  are  marginally  sig- 
nificant for  the  third  lowest  ranked  event,  and  none  of  the  groups  ranked  the 
' events  such  that  the  difference  is  significant  for  the  second  highest  ranked  event. 
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The  above  results  indicate  that  the  personnel  interviewed  per- 
ceived that  the  ADTP  assessment  policy  will  most  likely  encourage  the  fol- 
lowing two  events:  (1)  "establishing  an  early  informational  linkage"  be- 
tween the  laboratories  and  the  product  divisions  and  (2)  "strong  product 
division  support"  of  a technology.  A review  of  the  subjective  comments 
received  during  the  interviews  suggests  that  an  "early  informational  link- 
age" will  result  from  better  communication  and  dialogue  between  the  lab- 
oratories and  the  product  divisions.  This  aspect  of  the  assessment  process 
is  discussed  in  the  "Communication  and  Dialogue"  section  of  this  chapter. 
The  subjective  comments  also  suggest  that  "strong  product  division  support" 
will  result  from  two  factors:  (1)  a formal  acknowledgement  of  a product 
division  need  and  (2)  resolving  disagreements  within  the  product  division 
which  thus  creates  a single  spokesman  for  the  product  division. 

The  findings  regarding  the  ranks  assigned  to  the  event  "planning 
activities  which  may  expose  problems"  are  less  conclusive.  The  rank  as- 
signed to  this  event  did  not  differ  statistically  from  those  assigned  to 
either  the  highest  or  the  lowest  ranked  events.  However,  the  mean  of  the 
ranks  tends  to  be  closer  in  value  to  the  mean  of  the  ranks  assigned  to  the 
lowest  ranked  event. 

The  analysis  also  showed  that  the  two  least  likely  events  to  be 
encouraged  were  perceived  to  be  as  follows:  (1)  "establishing  that  the 
technology  is  sufficiently  mature"  and  (2)  "providing  a forum  for  problem 
resolution."  The  subjective  comments  srggest  that  some  personnel  feel  that 
the  product  divisions  do  not  have  enough  time  to  review  the  technology  to 
determine  if  it  is  mature.  Also,  some  of  the  people  interviewed  suggested 
that  establishing  that  the  technology  is  mature  sometimes  becomes  a sub- 
jective process.  The  recognition  of  these  factors  by  the  personnel 
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interviewed  could  have  resulted  in  the  lower  ranks  assigned  to  thi'-,  event. 

The  personnel  interviewed  offered  no  subjective  comments  regarding  the 
"provide  a forum"  event. 

When  the  above  findings  are  considered  in  the  context  of  the 
findings  of  research  efforts  discussed  in  Chapter  II,  it  can  be  concluded 
that  the  balance  between  the  push  of  technology  and  the  pull  of  the  product 
division  need  is  likely  to  be  enhanced  by  the  assessment  process.  The  Proj- 
ect Hindsight  team  concluded  that  in  the  majority  of  the  technological 
events  transferred  to  weapon  systems  that  the  need  was  first  established 
by  the  applications  engineering  group  (ODDR&E,  1969:47).  Furthermore,  the 
comments  made  by  the  Syracuse  Research  Corporation  study  group  indicate 
that  the  most  successful  cases  of  technology  transfer  occurred  when  a 
problem  solution  was  being  sought  and  that  the  balance  between  push  and 
pull  was  accomplished  by  an  early  informational  linkage  (Teich,  1974:42,  M), 
Difference  Between  Demographic  Groupings.  The  statistical 
hypothesis  that  there  tends  to  be  no  difference  between  the  responses 
given  by  the  various  demographic  groups  in  answer  to  this  question  was 
tested  using  the  Smirnov  two-sample  test  identified  in  Chapter  III.  This 
test  was  performed  for  each  of  the  five  technology  transfer  aiding  events, 
and  the  responses  obtained  from  two  demographic  groupings  were  compared 
during  each  test.  In  all,  six  different  pairs  of  demographic  groupings 
were  considered:  (1)  All  Laboratory  and  All  Product  Division,  (2)  Labora- 
tory A and  Laboratory  B,  (3)  Product  Division  Organization  A and  Product 
Division  Organization  B,  (4)  GS-15  Lt  Col  & Above  and  GS-14  Maj  & Below, 

(5)  Over  20  Years  and  Under  20  Years,  and  (6)  More  Than  2 Years  and  Less 
Than  2 Years.  The  combination  of  the  six  pairs  of  demographic  groupings 
and  five  aiding  events  resulted  in  thirty  separate  statistical  hypotheses 
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being  tested.  A significant  difference  between  the  responses  given  was 
not  found  for  any  of  the  thirty  combinations  considered.  It  is  folt  t.hat 
this  finding  corroborates  the  finding  that  there  is  agreement  among  those 
interviewed  regarding  which  events  are  likely  to  be  encouraged  by  the 
assessment  policy. 

Technology  Transfer  Barriers  (Questions  ]_  and  8) 

General.  The  proposed  barriers  to  technology  transfer 
that  those  interviewed  were  asked  to  consider  are  defined  in  Table  II 
(see  Chapter  III),  The  variation  in  mean  responses  to  these  questions 
suggests  that  the  personnel  interviewed  perceived  that  the  proposed  bar- 
riers to  technology  transfer  are  not  equally  significant  for  both  con- 
ditions—before  and  after  the  ADTP  assessment  policy  went  into  effect. 
Moreover,  a decrease  in  the  mean  responses  was  observed,  and  this  sug- 
gests that  the  assessment  process  was  perceived  to  reduce  the  signifi- 
cance of  the  barriers. 

Perception  of  Barriers 

Agreement.  Overall  there  tends  to  be  agreement 
among  those  interviewed  regarding  the  perceived  significance  of  the  tech- 
nology transfer  barriers.  Also,  there  generally  tends  to  be  agreement 
among  the  personnel  within  each  of  the  demographic  groupings  considered. 
These  findings  result  from  testing  the  statistical  hypotheses  shown  in 
Figure  6.  This  figure  is  constructed  in  the  same  manner  as  Figure  5, 
and  the  level  of  significance  (&)  values  were  determined  from  the  coef- 
ficient of  concordance  test.  The  broken  bars  represent  the  value  of  a 
computed  for  the  response  to  Question  7 (before  ADTP  assessments);  the 
solid  bars  represent  the  values  of  a computed  for  the  response  to  Ques- 
tion 8 (after  ADTP  assessments). 
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DEMOGRAPHIC  GROUPING 


|-  SIGNIFICANT  AGREEMENT 


All  Personnel 
n *=  38 


All  Laboratories 
n « 17 

Product  Division  (All 
Organizations)  n = 17 

Laboratory  A Only 
n * 6 


Jill 


Laboratory  B Only 
n « 9 

Product  Division  (Org  A 
Only)  n = 8 

Product  Division  (Org  B 
Only)  n * 9 

GS-15/Lt  Col  and  Above 
n * 13 

GS-14/Maj  and  Below 
n * 21 

Over  20  Yrs  Experience  in 
R&D  Field  n = 19 

Under  20  Yrs  Experience  in 
R&D  Field  n = 15 

More  Than  2 Yrs  in  Present 
Job  Assignment  n **  21 

Less  Than  2 Yrs  in  Present 
Job  Assignment  n = 13 

Question  8 HHBIBHI 
(After  ADTP  Assessments) 
Question  7 BHHMB 
(Before  ADTP  Assessments) 


,uui  iu 


LEVEL  OF  SIGNIFICANCE  ~ 5 
(CRITICAL  VALUE  .05) 


Hnt  There  is  no  agreement  among  those  interviewed  regarding  the 

0 significance  of  the  proposed  technology  transfer  barriers. 

R.  i There  tends  to  be  agreement  among  those  interviewed  regarding 

1 the  significance  of  the  proposed  technology  transfer  barriers. 


Figure  6.  Results  of  Coefficient  of  Concordance  Tests— 
Technology  Transfer  Barriers  (Questions  7 & 8) 
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The  results  shown  in  Figure  6 indicate  that  the  personnel  assigned 
to  laboratory'  A did  not  exhibit  significant  agreement  regarding  the  ranking 
of  the  technology  transfer  barriers  when  the  conditions  that  were  perceived 
to  have  existed  before  the  ADTP  assessment  policy  went  into  effect  were 
considered.  The  mean  of  the  ranks  assigned  to  the  proposed  barriers  are 
listed  in  Table  F-II  as  6.67,  A. 83,  4.75,  A. 50,  A. 25,  A. 00,  A. 00,  and  3.00. 
It  can  be  seen  that  except  for  the  largest  and  smallest  of  these  ranks  the 
values  are  grouped  near  the  center  of  the  ranking  scale.  In  fact,  the 
average  difference  between  the  mid-ranks  is  only  .17.  This  value  can 
be  contrasted  with  the  value  of  .67  obtained  when  the  same  computations 
are  performed  using  the  mean  of  its  ranks  assigned  by  the  laboratory  B 
personnel  who  did  exhibit  agreement  regarding  the  ranking  of  the  barriers. 
Thus,  the  lack  of  agreement  can  be  attributed  to  disagreement  in  the 
ranking  of  the  barriers  associated  with  the  six  mid-ranks  listed  above. 

The  results  shown  in  Figure  6 also  indicate  that  personnel  as- 
signed to  the  product  division  organization  A did  not  agree  on  the  ranks 
assigned  to  the  proposed  technology  transfer  barriers.  An  analysis  sim- 
ilar to  that  performed  for  the  laboratory  A personnel  repeals  that  the 
disagreement  is  also  with  respect  to  the  ranks  as  assigned  to  the  mid- 
ranked  barriers.  The  average  difference  in  the  ranks  assigned  to  these 
mid-ranks  was  the  same  (.24)  for  both  the  before  and  after  conditions. 

As  a final  comment  on  those  demographic  group’  gs  which  failed 
to  agree  on  the  ranking  of  the  technology  transfer  barriers,  note  that 
the  same  two  groupings  failed  ti  agree  on  the  ranking  of  the  technology 
transfer  aiding  events.  It  is  felt  that  this  fact  supports  the  recommen- 
dation of  further  study  diocussel  in  the  section  entitled  "Technology 
Transfer  Aiding  Events." 
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Significance  of  Barriers.  Estimates  of  the  perceived 
significance  of  the  eight  technology  transfer  barriers  were  made  by  as- 
signing ranks  to  those  barriers  as  described  in  Chapter  III  (see  Table  III). 
The  ranks  assigned  to  the  barriers  are  shown  in  Tables  IX  and  X.  Table  IX 
shows  the  values  assigned  for  the  conditions  perceived  to  exist  before 
the  ADTP  assessments,  and  Table  X shows  the  values  assigned  for  the  con- 
ditions perceived  to  exist  after  the  ADTP  assessments.  These  tables  are 
constructed  in  the  same  manner  as  Table  VIII.  A table  entry  of  "8"  indi- 
cates the  barrier  ranked  the  most  significant,  and  an  entry  of  "1"  indi- 
cates the  barrier  ranked  the  least  significant.  The  barriers  are  listed 
in  order  of  decreasing  significance  as  determined  by  the  mean  ranks  as- 
signed by  the  All  Personnel  demographic  grouping.  Thus,  the  results  shown 
in  Tables  DC  and  X indicate  the  estimated  preferential  order  of  the  bar- 
riers assigned  by  the  various  demographic  groups.  However,  it  should  be 
noted  that  the  estimated  order  shovn  might  well  change  if  anoth  _-r  series 
of  interviews  was  to  be  conducted  unless  the  difference  between  the  ranks 
is  shown  to  be  statistically  significant.  The  level  of  significance  is 
shown  by  asterisks  as  noted  in  the  tables. 

When  the  conditions  perceived  to  exist  before  ADTP  assessments 
(Table  DC)  are  considered,  it  can  be  seen  that  the  difference  between 
the  rank  assigned  to  the  highest  ranked  barrier  and  that  assigned  to  the 
lowest  ranked  barrier  was  found  to  be  statistically  significant  for  ten 
of  the  demographic  groupings  and  marginally  significant  for  one  demographic 
grouping.  The  level  of  significance  increases  when  the  second  and  third 
lowest  ranked  barriers  are  compared  to  the  highest  ranked  barrier,  and 
it  can  be  seer,  that  none  of  the  demographic  groups  ranked  the  barriers 
so  that  the  difference  is  significant  for  the  fourth  lowest  ranked  barrier. 
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The  results,  shown  in  Table  X (after  ADTP  assessments),  follow 
the  same  general  pattern  as  those  shown  in  Table  IX.  However,  it  can  be 
seen  that  for  these  conditions  the  differences  obtained  for  the  third 
lowest  ranked  barriers  are  more  significant  than  those  obtained  for  the 
conditions  perceived  to  exist  before  ADTP  assessments.  Five  of  the  third 
lowest  ranked  barriers  were  found  to  be  statistically  different  from  the 
highest  ranked  barrier.  Whereas  for  the  condition  perceived  to  exist 
before  the  ADTP  assessments  the  third  lowest  ranked  barriers  were  found 
to  be  marginally  different  from  the  highest  ranked  barrier. 

The  above  results  indicate  that  the  personnel  interviewed  per- 
ceived "inertia  barriers,"  "time  barriers,"  and  "cost  barriers"  to  be 
more  significant  than  "technology  barriers"  and  "geographic  distance." 
This  perception  is  true  for  the  conditions  perceived  to  exist  both  before 
and  after  the  ADTP  assessment  policy  went  into  effect.  The  subjective 
comments  received  during  the  interviews  suggest  that  the  higher  perceived 
significances  of  the  first  three  barriers  may  be  due  to  the  performance, 
sohedule,  and  cost  tradeoffs  that  must  be  made  during  the  weapon  system 
acquisition  process.  The  following  excerpts  from  these  comments  illus- 
trate this  suggestion. 

1.  The  SPO  director  is  the  big  factor  [^with  respect 
to  using  an  ADTP  generated  te''*':.clogyJ.  It  depends  on 
how  he  sees  the  risk  involved,  the  cost,  and  schedule 
impact. 

2.  The  labs  do  not  necessarily  have  all  the  solu- 
tions; in  fact,  the  contractors  will  stress  a solution 
which  may  have  a little  less  performance  and  risk,  but 
is  a lot  cheaper. 

3 . The  tiirr.G  bmrrioT  is  iispcrt^int*  The  technology 
window  is  pretty  narrow  sometimes. 

The  comments  offered  regarding  "technology  barriers"  generally  expressed 
the  idea  that  program  authorization  and  budgeting  coordination  cycles 
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will  eliminate  those  ADTPs  that  are  not  (supported  by  a suitable  technology 
base.  The  availability  of  the  telephone  was  usually  cited  as  the  reason 
for  "geographic  distance"  being  ranked  low. 

The  findings  regarding  the  barrier  "lack  of  incentive  structure" 
are  less  conclusive.  This  barrier  was  generally  ranked  as  being  marginally 
different  from  the  highest  ranked  barrier  for  the  conditions  perceived 
to  exist  befora  ADTP  assessments  went  into  effect  but  was  ranked  as  not 
being  different  for  the  conditions  perceived  to  exist  after  ADTP  assess- 
ments went  into  effect.  This  increase  in  the  ranking  may  be  due  to  the 
decrease  in  the  ranks  assigned  to  "transfer  management  structure"  and 
"communication  brxriers." 

Effect  of  ADTP  Assessments.  The  decrease  in  the  ranks  as- 
signed to  "transfer  .nanagemeiit  structure"  and  "communication  barriers"  are 
an  indication  that  the  personnel  interviewed  perceived  that  the  assess- 
ment process  will  strongly  affect  these  two  barriers.  This  indication  is 
corroborated  by  the  results  obtained  from  using  the  sign  test  (see  Chap- 
ter III).  The  null  hypothesis  that  the  barriers  were  perceived  to  be 
less  significant  before  the  ADTP  assessment  policy  went  into  effect  was 
tested  using  the  responses  from  the  All  Personnel  demographic  grouping. 

The  results  of  this  test  indicate  the  perceived  effect  of  ADTP  assessments 
on  the  technology  transfer  barriers  and  are  shown  in  Table  XI. 

Table  XI  is  constructed  such  that  the  entries  in  the  first  three 
table  columns  are  the  number  of  personnel  who  perceived  that  a given  har- 
rier was  less  significant  before  ADTP  assessments,  less  significant  after 
ADTP  assessments,  and  of  the  same  significance  for  both  before  and  after 
the  ADTP  assessments.  The  entries  in  the  last  table  column  are  values 
computed  for  the  level  of  significance  using  the  conservative  approach 
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described  in  Chapter  III.  The  table  rows  are  the  proposed  barriers  to 
technology  transfer. 

The  results  shown  in  Table  XI  indicate  that  the  null  Hypothesis 
stated  above  can  be  rejected  at  significant  levels  less  than  .05  for 
"communication  barriers."  Thus,  it  can  be  concluded  that  the  personnel 
interviewed  generally  perceived  that  this  barrier  was  more  significant 
prior  to  establishing  the  ADTP  assessment  policy  than  after.  Also,  note 
that  the  null  hypothesis  can  be  rejected  at  a significant  level  of  .072 
for  "transfer  management  structure."  The  results  shown  in  Table  XI  also 
clearly  indicate  that  the  null  hypothesis  cannot  be  rejected  for  the 
remaining  six  proposed  barriers.  Furthermore,  the  number  of  people  who 
indicated  that  they  perceived  no  change  in  the  significance  of  the  barriers 
is  larger  than  the  number  who  indicated  that  the  barrier  was  less  signif- 
icant before  ADTP  assessments.  Therefore,  these  results  suggest  that 
those  interviewed  perceived  that  the  assessment  process  would  not  affect 
the  significance  of  the  remaining  six  barriers. 

Recommendation.  The  above  results  indicate  that  the  ADTP 
assessment  process  was  not  perceived  to  significantly  affect  six  of  the 
proposed  barriers  to  technology  transfer,  .’it  is  therefore  recommended 
that  research  on  these  barriers  to  technology  transfer  be  continued.  A 
follow-on  thesis  effort  could  be  devoted  to  a study  of  the  three  barriers 
considered  to  be  the  mojt  significant.  Such  an  effort  might  lead  to  a 
better  understanding  of  the  chief  characteristics  of  these  barriers. 

Difference  Between  Demographic  Groups.  The  statistical 
hypothesis  that  there  tends  to  be  no  difference  between  the  responses 
given  by  the  various  demographic  groups  in  answer  to  Questions  7 and  8 
was  tested.  The  Smirnov  two-sample  test  was  used  in  performing  this 
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analysis,  and  the  procedure  followed  was  the  same  as  that  used  in  performing 
two-sample  tests  when  analyzing  the  responses  obtained  in  answer  to  Ques- 
tion 6.  The  two  instances  found  for  whi:h  a significant  difference  exists 
in  the  response  to  these  questions  are  summarized  in  Table  XII, 

The  values  for  the  level  of  significance  shown  in  Table  XII  in- 
dicate that  the  laboratory  and  product  division  personnel  tend  to  give 
statistically  different  answers  for  "lack  of  incentive  structure"  in  re- 
sponse to  Question  7,  but  not  in  response  to  Question  8.  The  means  of 
the  responses  given  for  each  of  the  groups  are  also  shown  in  Table  XII 
for  both  Questions  7 and  8.  These  values  indicate  that  the  laboratory 
personnel  feel  that  this  barrier  is  less  significant  than  the  product 
division  personnel.  Moreover,  the  values  of  the  difference  in  the  mean 
responses  are  less  for  Question  8 than  for  Question  7,  This  indicates 
that  the  product  division  personnel  generally  felt  that  the  assessment 
process  would  reduce  the  significance  of  this  barrier  more  than  the  lab- 
oratory personnel  did. 

The  results  shown  in  Table  XII  also  indicate  that  the  personnel 
working  in  laboratory  A and  laboratory  B were  found  to  have  given  dif- 
ferent responses  regarding  the  terrier  "transfer  Management  structure." 
Moreover,  this  difference  was  not  found  to  be  significant  for  the  responses 
to  Question  8.  These  differences  and  the  fact  that  the  means  of  the  re- 
sponses given  by  the  laboratory  A personnel  are  less  than  the  means  of 
those  given  by  the  laboratory  B personnel  suggest  that  the  barrier  was 
perceived  to  be  less  significant  to  the  personnel  working  in  laboratory  A, 

Communication  and  Dialogue 

The  analysis  of  the  responses  to  the  questions  regarding  technology 
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transfer  revealed  that  the  personnel  interviewed  perceived  that  the  ADTP 
assessment  nrocess  would  affect  communication  and  dialogue  between  the 
laboratories  and  the  product  division.  It  was  found  that  the  event  "es- 
tablish early  information  linkage"  was  perceived  likely  to  be  encouraged 
by  the  assessment  process.  Moreover,  it  was  found  that  the  personnel 
perceived  that  the  assessment  process  would  reduce  "communication  barriers." 

The  data  analyzed  in  this  section  of  the  thesis  were  obtained  from  questions 
designed  to  further  investigate  laboratory  and  product  division  communi- 
cation and  dialogue.  Interview  Questions  9,  10,  11,  12,  13,  and  14  were 
devoted  to  this  end  and  were  designed  to  measure  four  specific  factors: 
advanced  development  program  visibility,  an  individual's  communication 
pattern,  adequacy  of  expected  communication,  and  anticipation  of  increased 
dialogue.  The  development  of  these  questions  is  presented  in  Chapter  III 
under  the  section  entitled  "Communication  and  Dialogue." 

Program  Visibility  (Questions  9.  and  10) 

General.  The  degree  of  visibility  of  ADTPs  that  can  be 
attributed  to  the  assessment  process  was  measured  through  interview  Ques- 
tions 9 and  10. 

Question  9:  Below  is  a list  of  advanced  development 

programs/projects  either  being  considered  or  currently 
ongoing  in  the  laboratories.  Please  place  a check  mark 
beside  a program/ project  if  you  learned  of  it  because 
of  the  ADTP  assessment  policy/ activity. 

Question  10a:  Can  you  think  of  any  other  advanc>_u  devel- 

opment programs/projects  that  you  now  know  about  but 
probably  would  not  have  learned  of  if  the  ADTP  assess- 
ment policy/activity  had  not  occurred? 

Question  10b:  If  more  than  one,  about  how  many? 

Seven  of  the  thirty-eight  personnel  interviewed  did  not  answer  these  two 

questions.  Six  of  those  who  did  not  answer  were  laboratory  personnel.  The 

( 

remaining  person  who  did  not  answer  was  a member  of  the  Othe : organization 
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category.  The  view  generally  expressed  by  the  laboratory  personnel  who 
declined  to  answer  these  questions  was  that  the  questions  should  be  an- 
swered only  by  product  division  personnel. 

Findings.  The  results  obtained  from  the  answers  to  these 
two  questions  suggest  that  the  assessment  process  does  not  vastly  increase 
the  visibility  of  ADTPs  among  laboratory  and  product  personnel.  Twelve 
(39%)  individuals  indicated  that  they  had  not  heard  of  any  of  the  ADTPs 
listed  in  Question  9 because  of  the  assessment  process  (see  Appendix  D). 
Also,  nineteen  (61%)  indicated  that  they  had  not  heard  of  any  ADTPs  in 
addition  to  those  listed  in  Question  9 because  of  the  assessment  process. 
Moreover,  only  four  persons  indicated  that  they  had  heard  of  more  than 
four  of  the  fifteen  programs  listed  in  Question  9 because  of  ADTP  assess- 
ments, and  only  three  persons  indicated  that  they  had  heard  of  more  than 
four  additional  programs  in  answer  to  Question  10b. 

Possible  Bias.  The  fact  that  the  assessment  process  had 
been  just  recently  established  when  the  interviews  were  conducted  most 
likely  biased  the  results  obtained  from  these  two  questions.  Many  of 
the  ADTPs  were  ongoing  when  the  assessment  process  was  implemented;  thus, 
people  had  the  opportunity  to  learn  of  these  programs  by  other  means  be- 
fore the  assessment  policy  was  established.  This  prior  knowledge  would 
result  in  fewer  programs  being  learned  of  because  of  the  assessment  pro- 
cess. Under  these  conditions,  it  is  felt  that  the  results  obtained  from 
Question  9 and  10  are  likely  to  be  biased  toward  more  negative  findings 
regarding  the  ability  of  the  assessment  process  to  make  visible  new  ad- 
vanced development  technology  programs  (ADTPs). 

Communication  Pattern  (Questions  11  and  12) 

General.  The  effect  of  the  ADTP  assessment  process  upon  an 
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individual's  communication  pattern  was  addressed  through  interview  Ques- 
tions 11  and  12, 

Question  11s  What  is  your  best  estimate  of  the  percent 
of  your  working  time  that  was  spent  communicating  with 
personnel  in  the  agencies  listed  below  regarding  new  ADTPs 
before  the  ADTP  assessment  requirement  went  into  effect? 

Question  12s  What  is  your  best  estimate  of  the  percent 
of  your  working  time  that  is  or  will  be  spent  communi- 
cating with  personnel  in  the  agencies  listed  below  re- 
garding new  ADTPs  now  that  the  ADTP  assessment  require- 
ment is  in  effect? 

The  agencies  listed  were  "higher  headquarters,"  'product  divisions,"  "lab- 
oratories," "AFLC,"  and  "operational  commands."  Also,  note  that  percent- 
age estimates  were  only  for  communications  regarding  new  ADTPs. 

Two  of  the  thirty-eight  personnel  interviewed  did  not  answer  these 
two  questions.  One  person  who  did  not  answer  was  assigned  to  laboratory  C, 
and  one  person  was  classified  into  the  Other  organization  category. 

The  interpretation  of  the  percentages  obtained  in  response  to 
Questions  11  and  12  was  modified  as  a result  of  the  initial  interviews. 

It  was  originally  intended  to  use  the  values  of  the  percentages  of  working 
time  given  in  response  to  these  questions  as  absolute  percentages,  but 
during  the  conduct  of  the  trial  interviews  it  was  found  that  people  had 
difficulty  in  estimating  these  percentages  in  absolute  terms.  The  re- 
searcher therefore  made  it  a practice  to  indicate  that  relative  percent- 
ages would  be  acceptable  as  answers  to  these  questions.  These  relative 
percentage?  were  then  used  to  assign  ranks  to  the  five  agencies  listed. 

The  agency  receiving  the  highest  percentage  was  assigned  the  highest 
numerical  rank,  and  the  agency  receiving  the  lowest  percentage  was  as- 
signed the  lowest  numerical  rani..  Also,  the  estimates  of  the  relative 
percentages  were  used  to  identic)  whether  communication  with  personnel 
in  these  agencies  're  perceived  to  have  increased,  stayed  the  same,  or 
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fJo{:r<-;if;c«I  b»**:au«»*  of  l.h«*  assessment  process.  The  result  of  this  modifi- 
cation of  the  interpretation  of  the  responses  received  in  answer  to 
these  questions  is  that  the  same  techniques  used  in  analyzing  the  re- 
sponses to  Questions  7 and  8 are  applicable  to  analyzing  the  response 
to  the  two  questions. 

Perception  of  Communication  Time 

Agreement.  The  null  hypothesis  of  no  agreement  among 
those  interviewed  regarding  the  relative  percentage  of  their  working  time 
spent  in  communicating  with  personnel  in  the  five  agencies  was  tested 
using  the  coof fit. lent  of  concordance  test.  Each  demographic  grouping  de- 
fined in  Table  V was  considered.  The  level  of  significance  computed  for 
this  test  and  the  specific  hypothesis  tested  are  shown  in  Figure  7.  This 
figure  is  constructed  in  the  same  manner  as  Figures  5 and  6.  It  can  be  seen 
that  personnel  in  each  of  the  thirteen  demographic  groupings  tend  to  exhibit 
agreement  in  assigning  the  relative  percentage  of  their  working  time  spent 
in  communicating  with  personnel  in  the  five  different  agencies. 

Preferred  Order.  Estimates  of  the  true  preferential 
order  assigned  to  the  five  agencies  by  the  various  demographic  groupings 
are  shown  in  Tables  XITI  and  XIV.  These  tables  are  constructed  in  the 
,.ame  manner  as  Tables  VIII,  IX,  and  X.  Table  XIII  shows  the  values 
assigned  for  the  conditions  perceived  to  exist  before  ADTP  assessments, 
and  Table  XIV  shows  the  values  assigned  for  the  conditions  perceived 
to  exist  after  ADTP  assessments.  A table  entry  of  "5"  indicates  the 
agency  that  the  personnel  interviewed  perceived  to  spend  the  highest 
percent  of  their  working  time  in  communication  with.  A table  entry  of 
"1"  indicates  that  the  agency  that  the  personnel  interviewed  perceived 
to  spend  the  least  amount  of  their  working  time  in  communication  with. 
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n * 36 
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n « 16 

Product  Division  (All 
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Laboratory  A Only 
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Only)  n » 8 

Product  Division  (Org  B 
Only)  n * 9 
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Figure  7,  Results  of  Coefficient  of  Concordance  Tests 
Communication  Time  (Questions  11  A 12) 
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It  should  be  noted  that  the  estimated  rank  order  of  the  agencies  shown 
in  Tables  XIII  and  XIV  might  well  change  if  another  set  of  interviews 
was  to  be  conducted,  unless  the  difference  between  the  ranks  is  shown 
to  be  statistically  significant.  The  level  of  significance  is  shown 
by  asterisks  as  noted  in  the  tables. 

When  the-  conditions  which  existed,  before  ADTP  assessments 
(Table  X.7II)  are  considered,  it  can  be  seen  that  the  difference  between 
the  rank  assigned  to  the  highest  ranked  agency  and  that  assigned  to  the 
lowest  ranked  agency  was  found  to  be  significant  at  the  .05  level  for  ten 
of  the  demographic  groupings.  Moreover,  this  difference  was  found  to  be 
significant  at  the  .1  level  for  the  remaining  three  demographic  groupings. 
Hie  difference  between  the  rank  assigned  to  the  second  lowest  ranked  agency 
and  that  assigned  to  the  highest  ranked  agency  was  found  to  be  significant 
at  the  .05  level  for  five  of  the  demographic  groupings  and  significant  at 
the  .1  level  for  one  demographic  grouping.  Only  one  demographic  grouping 
ranked  the  agencies  so  that  the  rank  assigned  to  the  third  lowest  ranked 
agency  w^s  statistically  different  from  that  assigned  to  the  highest  ranked 
agency.  None  of  the  demographic  groupings  ranked  the  agencies  so  that  the 
difference  is  significant  for  the  second  highest  ranked  agency. 

When  the  conditions  which  existed  after  ADTP  assessments 
(Table  XIV)  are  considered,  it  c.n  be  seen  that  the  results  follow  the 
same  general  pattern  as  those  shown  in  Table  XIII.  However,  it  can  also 
be  seen  that  the  differences  between  the  ranks  assigned  to  the  highest 
ranked  agency  are  generally  more  significant  when  these  conditions  are 
considered.  Also,  note  that  one  demographic  grouping  (P.  D. ) ranked  the 
agencies  so  that  the  difference  between  the  ranks  assigned  to  the  highest 
agency  and  the  second  highest  ranked  agency  was  statistically  significant. 
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The  results  shown  in  Tables  XIII  and  XIV  confirm  the  results 
expected  by  intuition.  For  the  conditions  perceived  to  exist  both  be- 
fore and  after  ADTP  assessments  the  '"laboratories, " "higher  headquarters," 
and  "product  divisions"  were  ranked  consistently  high,  and  "operational 
commands"  and  "AFLC"  were  ranked  consistently  low.  Thus,  the  personnel 
interviewed  perceived  that  they  spend  more  of  their  working  time  com- 
municating about  new  ADTPs  with  personnel  in  "laboratories,"  "product 
divisions,"  and  "higher  headquarters"  than  they  spend  in  communicating 
with  personnel  in  "operational  commands"  and  "AFLC." 

Effect  of  ADTP  Assessments.  The  null  hypothesis  that  the 
personnel  interviewed  perceived  that  they  spent  more  of  their  working 
time  in  communication  with  personnel  in  an  agency  regarding  new  ADTPs 
before  the  assessment  policy  went  into  effect  was  tested.  This  test  was 
performed  for  each  of  the  five  agencies  listed  in  Questions  11  and  12, 
and  both  organizational  and  nonorganizational  demographic  groupings  were 
considered.  The  results  obtained  from  these  tests  for  the  All  Personnel, 
All  Product  Division  Organizations,  and  All  Laboratories  demographic 
groupings  are  shown  in  Tables  XV,  XVI,  and  XVII  respectively.  It  was 
found  that  the  values  of  the  level  of  significance  computed  for  the  non- 
organizational demographic  groupings  were  larger  than  the  critical  value. 
Thus,  the  null  hypotheses  could  not  be  rejected  for  these  tests.  These 
results  are  therefore  not  tabulated. 

The  construction  of  Tables  XV,  XVI,  and  XVII  is  similar  to 
that  of  Table  XI.  The  entries  in  the  first  three  columns  of  each  table 
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The  entries  in  the  fourth  table  column  are  the  values  computed  for  the 
level  of  significance  when  testing  the  null  hypotheses  stated  above. 

The  results  obtained  from  the  before-and-af ter  sign  tests  in- 
dicate that  the  personnel  interviewed  did  in  fact  perceive  that  the 
assessment  policy  either  has  or  will  change  their  communication  pattern. 

In  order  to  perceive  this  finding  the  responses  provided  by  laboratory 
and  product  division  personnel  must  be  considered  separately.  When  the 
responses  from  all  of  the  personnel  interviewed  are  considered  together, 
the  results  shown  in  Table  XV  indicate  that  the  null  hypothesis  cannot 
be  rejected.  However,  if  the  responses  received  from  the  product  divi- 
sion personnel  (Table  XVI)  and  the  laboratory  personnel  (Table  XVII)  are 
considered  separately,  the  results  indicate  that  the  null  hypothesis 
can  be  rejected  for  either  the  laboratory  or  product  division  agencies. 

The  rejection  of  these  hypotheses  allows  for  the  following  conclusions: 

(1)  the  laboratory  personnel  interviewed  perceived  that  more  of  their 
working  time  will  be  spent  communicating  about  new  ADTPs  with  product 
division  personnel  now  that  the  assessment  policy  is  in  effect,  and 

(2)  the  product  division  personnel  interviewed  perceived  that  more  of 
their  working  time  will  be  spent  communicating  about  new  ADTPs  with 
laboratory  personnel  now  that  the  assessment  policy  is  in  effect.  Thus, 
the  results  indicate  that  the  communication  between  the  laboratory  and 
product  division  personnel  will  be  enhanced  by  the  ADTP  assessment  process. 

Difference  Between  Demographic  Groups.  The  statistical 
hypothesis  that  there  tends  to  be  no  difference  in  the  responses  given 
by  the  various  demographic  groupings  in  answer  to  Questions  11  and  12 
was  tested  in  the  same  manner  that  this  hypothesis  was  tested  for  Ques- 
tions 6,  7,  and  8.  Each  agency  was  considered,  and  each  of  the  six  pairs 
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of  demographic  groupings  defined  earlier  were  considered.  Five  cases 
were  found  for  which  there  were  significant  differences  in  the  responses 
given  by  two  demographic  groupings.  In  each  case,  the  demographic  groupings 
which  gave  the  responses  were  the  All  Laboratories  and  All  Product  Division 
Organizations.  Furth more,  four  of  the  differences  in  responses  given 
in  answer  to  Questions  11  and  12  occurred  for  the  same  two  agencies — 

"higher  headquarters"  and  "product  divisions."  The  significant  differences 
found  are  summarized  in  Table  XVIII. 

It  can  be  seen  from  the  results  shown  in  Table  XVIII  that  labor- 
atory and  product  division  personnel  tend  to  give  statistically  different 
answers  regarding  the  time  spent  communicating  with  personnel  in  "higher 
headquarters"  about  new  ADTPs.  Furthermore,,  the  means  of  the  responses 
suggest  that  laboratory  personnel  tend  to  communicate  more  with  personnel 
in  "higher  headquarters"  about  new  ADTPs  than  product  division  personnel. 

Also,  note  that  this  tendency  was  not  perceived  to  be  changed  by  the  assess- 
ment process.  The  differences  noted  could  be  due  \o  the  fact  that  funding 
for  new  ADTPr.  comes  from  higher  headquarters.  Thus,  laboratory  personnel 
would  be  more  inclined  to  communicate  with  these  agencies  regarding  new  ADTPs. 

The  results  shown  in  Table  XVIII  also  indicate  that  the  laboratory 
and  product  division  personnel  tend  to  give  statistically  different  answers 
regarding  the  time  spent  communicating  with  personnel  in  "product  divisions" 
and  "operational  commands"  about  new  ADTPs.  The  means  of  the  responses 
suggest  that  the  product  division  personnel  tend  to  communicate  more  with 
people  in  "product  divisions"  than  laboratory  personnel,  but  that  they  tend 
to  communicate  less  with  people  in  "operational  commands"  than  laboratory 
personnel.  The  difference  in  the  responses  is  not  significant  at  a reason- 
able level  of  significance  for  the  conditions  perceived  to  exist  after 
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the  ADTP  assessment  policy  went  into  effect.  This  change  in  significance 
is  due  primarily  to  the  fact  that  four  product  division  personnel,  who 
indicated  zero  percent  in  answer  to  Question  11,  indicated  that  some  per- 
centage of  their  working  time  would  be  spent  in  communication  with  per- 
sonnel in  the  "operational  commands"  in  answer  to  Question  12. 

Adequacy  of  Expected  Communications  (Question  13) 

General.  The  adequacy  of  the  amount  of  time  that  personnel 
will  spend  in  communication  with  either  laboratory  or  product  division  per- 
sonnel regarding  new  ADTPs  can  be  measured  by  considering  the  responses 
given  in  answer  to  Question  13. 

Question  13s  How  much  of  your  working  time  should  be 
spent  communicating  with  personnel  in  the  laboratory 
(product  division)  in  order  for  you  to  stay  aware  of 
status  of  advanced  development  programs  (systems) 
which  are  relevant  to  your  job? 

The  responses  given  in  answer  to  this  question  can  be  compared  to  appro- 
priate responses  given  in  answer  to  Question  12,  For  example,  the  per- 
centage estimated  by  a person  working  in  a laboratory  in  answer  to  this 
question  can  be  compared  to  the  percentage  he  estimated  in  Question  12 
for  "product  divisions."  When  the  responses  are  compared  in  this  manner, 
inferences  can  be  made  regarding  whether  the  personnel  interviewed  per- 
ceived that  communication  will  be  more  or  less  than  should  be.  The 
same  personnel  who  answered  Questions  11  and  12  answered  this  question. 

Findings.  The  statistical  hypothesis  that  the  personnel 
interviewed  perceived  that  less  time  should  be  spent  communicating  than 
will  be  spent  was  tested  using  the  sign  test.  The  responses  given  by 
laboratory  personnel  and  product  division  personnel  were  considered 
separately.  Furthermore,  the  responses  giiren  by  the  personnel  in  each 
of  the  organizational  subgroupings  (see  Table  V)  were  also  considered 
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separately.  The  results  of  these  tests  are  shown  in  Table  XIX. 

Table  XIX  is  constructed  in  essentially  the  same  manner  as 
Tables  XV,  XVI,  and  XVII.  The  table  entries  in  the  first  three  columns 
indicate  the  number  of  people  who  answered  in  the  manner  described  by 
the  table  column  heading.  The  values  of  level  of  significance  computed 
are  shown  in  the  last  two  table  columns.  The  level  of  significance  values 
listed  under  the  leading  "Conventional  Sign  Test"  were  computed  by  dis- 
carding those  responses  which  indicated  the  amount  of  time  spent  in  com- 
municating as  being  what  it  should  (i.e.,  the  ties).  The  level  of  signif- 
icance values  listed  under  the  heading  "Conservative  Sign  Test"  were  com- 
puted by  considering  the  ties  as  discussed  in  Chapter  III  under  the  heading 
entitled  "Data  Reduction." 

It  can  be  seen  from  the  results  shown  in  Table  XIX  that  the  null 
hypothesis  stated  above  cannot  be  rejected  for  any  of  the  organizational 
groupings  considered  if  the  conservative  values  of  the  level  of  signifi- 
cance are  accepted.  However,  note  that  the  responses  obtained  tend  toward 
the  conclusion  that  the  amount  of  communication  will  be  less  than  it 
should  be.  Also,  if  the  values  computed  using  the  conventional  sign 
test  are  accepted,  the  null  hypothesis  can  be  rejected  for  the  All  Product 
Division  Organizations  and  Product  Division  Organization  A demographic 
groupings.  The  conclusion  drawn  from  the  above  considerations  is  that 
the  responses  obtained  tend  to  suggest  that  the  personnel  interviewed 
perceived  tnat  the  amount  of  communication  will  be  less  than  it  should 
be;  however,  acceptance  of  the  conclusion  cannot  be  supported  statistically. 

Increased  Dialogue  (Question  14) 

General.  The  anticipation  of  increased  dialogue  by  laboratory 
and  product  division  personnel  was  measured  by  interview  Question  14. 
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Question  14 t Do  you  think  that  the  ADTP  assessment  policy 
will  improve  the  dialogue  between  yourself  and  the  product 
division  (laboratory)? 

All  laboratory  and  product  division  personnel  interviewed  answered  this  ques- 
tion} however,  one  individual  in  the  Other  organization  category  did  not. 

Two  answer  choices  were  provided— "yes"  and  "no."  The  responses  given 
in  answer  to  this  question  were  analyzed  by  constructing  confidence  inter- 
vals for  the  percentage  of  "yes"  responses  and  by  contingency  table  tests. 

Findings.  The  histogram  shown  in  Appendix  D indicates  that 
thirty-one  (83.8%)  of  the  personnel  interviewed  answered  "yes"  to  this 
questions.  This  suggests  that,  in  general,  those  interviewed  perceived 
that  the  assessment  policy  will  improve  the  dialogue  between  themselves 
and  personnel  in  the  counterpart  agencies.  However,  it  was  felt  that 
additional  information  could  be  obtained  regarding  the  issue  addressed 
by  Question  14  if  the  data  were  to  be  examined  in  more  detail.  Accord- 
ingly, 95%  confidence  intervals  for  the  percentage  of  "yes"  responses 
were  computed.  All  of  the  organizational  and  nonorganizational  demo- 
graphic groupings  were  considered.  Confidence  intervals  were  not  com- 
puted using  the  responses  given  by  the  suborganizational  demographic 
groupings  (i.e..  Laboratory  A,  Laboratory  D,  etc.)  because  of  the  small 
size  of  these  groupings.  The  results  of  these  computations  are  jhown  in 
Figure  8.  A linear-percent  scale  is  used  to  plot  the  results,  and  the 
length  of  the  bar  represents  the  length  of  the  confidence  interval.  The 
bar  begins  at  the  lower  bound  of  the  estimate  for  the  percentage  of  per- 
sonnel who  will  answer  "yes"  and  extends  to  the  upper  bour.d  for  the  esti- 
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It  was  found  that  the  laboratory  personnel  tend  to  be  more  nega- 
tive in  their  answers  to  this  question  than  the  product  division  pe*'  - 1. 
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The  results  shown  in  Figure  8 indicate  that:  the  interval  computed  for  the 
laboratory  personnel  extends  between  28 % and  88%,  and  the  interval  for 
the  product  division  personnel  is  a single  point  (100%).  All  seventeen 
of  the  product  division  personnel  interviewed  answered  "yes."  Further- 
more, the  difference  between  the  responses  given  by  the  laboratory  and 
product  division  personnel  was  found  to  be  statistically  significant 
(a  = .007)  using  the  contingency  table  analytical  technique. 

The  subjective  comments  provided  by  the  laboratory  personnel  who 
answered  "no"  to  Question  14  were  reviewed  in  order  to  determine  if  these 
personnel  expressed  a consensus  of  opinion.  The  comments  provided  by 
those  personnel  are  listed  below. 

1.  Our  communications  with  the  product  division  per- 
sonnel is  [sic]]  very  good  now.  The  new  policy  may  work  if 
the  product  division  players  are  not  changed.  If  so,  the 
communications  channels  will  remain  the  same  [as  they  are 
now]]. 

2.  I haven’t  found  the  development  planners  partic- 
ularly informative  on  what  the  real  needs  are  in  the  prod- 
uct divisions. 

3.  It  has  served  to  make  everyone  defensive. 

4.  There  should  be  extensive  dialogue  with  the  prod- 
uct division,  but  I disagree  with  the  current  assessment 
policy. 

It  can  been  seen  that  three  different  opinions  are  expressed  by  these 
comments.  (1)  Existing  channels  for  dialogue,  which  bypass  the  devel- 
opment planning  organization,  are  sufficient.  (2)  Assessments  will  make 
laboratory  personnel  more  defensive  regarding  their  programs.  (3)  The 
concept  of  ADTP  assessment  is  not  desirable. 

Upon  further  review  of  the  data  it  was  also  found  that  the  Lab- 
oratory A and  Laboratory  B personnel  tended  to  give  different  answers  in 
response  to  this  question.  Four  of  the  six  Laboratory  A personnel  answered 
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"no"  to  the  question}  whereas  two  of  the  nine  Laboratory  B personnel  an- 
swered "noV"  This  difference  in  the  responses  was  found  to  be  significant 
at  the  .085  level  using  the  contingency  table  test,  Thus,  it  appears  that 
the  difference  between  the  laboratory  and  product  division  personnel  re- 
sponses may  be  attributed  to  the  answers  given  by  the  Laboratory  A per- 
sonnel. It  is  felt  that  this  finding  might  indicate  a negative  reaction 
to  the  assessment  process  because  the  Laboratory  A personnel  interviewed 
were  more  closely  involved  with  the  in-depth  assessment  process  than  the 
Laboratory  B personnel  interviewed.  However,  the  reader  is  reminded  that 
this  inference  is  based  upon  interviews  with  only  six  Laboratory  A per- 
sonnel. It  should  therefore  be  validated  by  further  research  before 
being  accepted. 

The  subjective  comments  offered  by  the  product  division  personnel 
illustrate  one  practice  which  might  degrade  the  effectiveness  of  the  as- 
sessment process.  Three  different  individuals  raised  the  issue  that 
within  a given  office  the  assessment  might  be  accomplished  by  only  one 
person.  Furthermore,  the  point  was  raised  that  an  annual  reassessment 
might  be  accomplished  by  an  individual  who  did  not  perform  the  orignial 
assessment  and  that  the  new  assessor  may  or  may  not  be  familiar  with  the 
conclusions  reached  during  the  first  assessment.  It  is  not  known  if  this 
situation  is  pervasive  throughout  the  product  division  organizations 
since  only  a small  number  of  people  were  interviewed.  However,  if  this 
situation  does  occur  often,  it  could  result  in  inconsistent  assessments 
and  would  most  likely  result  in  less  than  the  desired  visibility  of  lab- 
oratory programs  within  the  product  division. 

Recommendation.  It  is  recommended  that  the  possibility  of 
the  above  situations  be  recognized  and,  if  required,  that  action  be  taken 
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to  minimize  the  negative  aspects  of  one-person-assessments  within  an 
office.  One  action  that  could  be  taken  is  to  insure  that  the  results 
of  the  assessments  are  disseminated  among  the  office  personnel.  Also, 
an  office  case  file  could  be  established  for  each  laboratory  program  as- 
sessed. These  files  would  provide  a record  of  all  past  comments  and  data 
considered  and  could  be  referenced  when  performing  future  assessments. 

Assessment  Tasks 

The  personnel  interviewed  were  asked  to  give  their  perception  of 
key  assessment  tasks  in  Questions  15  and  16.  Laboratory  personnel  answered 
Question  15,  and  product  division  personnel  answered  Question  16.  Each 
question  contained  two  parts.  In  the  first  part  of  the  questions  those 
interviewed  were  asked  to  give  their  perception  of  the  difficulty  of  ac- 
complishing the  assessment  tasks,  and  in  the  second  part  they  were  asked 
if  they  thought  that  product  divisions  have  enough  time  and  manpower  to 
devote  to  the  assessment  tasks.  The  development  of  these  questions  is 
discussed  in  Chapter  III  under  the  section  entitled  "Assessment  Tasks." 
Difficulty  of  Assessment  Tasks  (Questions  15a  and  16a) 

General.  Questions  15a  and  16a  were  stated  as  follows: 

Question  15a:  If  you  work  in  a laboratory,  how  difficult 

is  it  (or  "would  it  be")  for  you  to  obtain  what  you  feel 
is  reliable  information  on  each  of  the  following  items  to 
include  in  the  technology  program  plan? 

Question  16a:  If  you  work  in  a product  division,  ho:-:  dif- 

ficult is  it  (or  "would  it  be")  for  you  to  perform  what 
you  feel  is  a reliable  assessment  of  an  ADTP  with  respect 
to  each  of  the  following  items? 

The  "items"  listed  were  "payoff,"  "schedules,"  "ADTP  cost  estimates," 
"priority,"  and  "alternate  approaches/decision  options." 

All  of  the  personnel  interviewed  answered  either  one  or  the 
other  of  these  questions.  However,  since  the  questions  addressed  the 
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assessment  task  from  an  organization-specific  perspective,  the  nonor- 
ganizational  demographic  groupings  were  not  considered  in  the  analysis. 
The  All  Personnel  demographic  grouping  also  was  not  considered  for  the 
same  reason. 

Perception  of  Task  Difficulty 

Agreement.  The  null  hypothesis  of  no  agreement  among 
those  interviewed  regarding  the  difficulty  of  the  assessment  task  was 
tested  using  the  coefficient  of  concordance  test.  The  results  of  the 
test  performed  are  shown  in  Figure  9.  This  figure  is  constructed  in 
the  same  manner  as  Figure  5. 

It  can  be  seen  from  the  results  shown  in  Figure  9 that  the  lab- 
oratory personnel  interviewed  tended  to  be  in  better  agreement  regarding 
the  difficulty  assigned  to  the  assessment  task  than  the  product  division 
personnel.  The  level  of  significance  for  the  All  Laboratory  and  Labora- 
tory A demographic  groupings  was  found  to  be  less  than  the  level  of  sig- 
nificance for  the  product  division  demographic  groupings.  The  personnel 
assigned  to  Product  Division  Organization  B were  also  found  to  be  in  less 
agreement  regarding  the  difficulty  of  the  assessment  task  than  the  other 
personnel  interviewed. 

Preferred  Order.  Estimates  of  the  true  preferential 
order  assigned  to  the  five  tasks  by  the  two  demographic  groupings  which 
did  exhibit  agreement  in  the  rankings  are  shown  in  Table  II.  It  can 
be  seen  that,  overall,  the  laboratory  personnel  interviewed  perceived 
that  obtaining  information  regarding  "payoff',’  and  "schedules"  was  less 
difficult  than  obtaining  information  regarding  the  other  tasks.  How- 
ever, note  that  the  difference  in  the  ranks  assigned  is  only  signifi- 
cant at  the  .1  level. 
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All  Laboratories 
n = 17 

Product  Division  (All 
Organizations)  n = 17 

Laboratory  A Only 
n = 6 

Laboratory  B Only 
n = 9 

Product  Division  (Org  A 
Only)  n = 8 

Product  Division  (Org  B 
Only)  n * 9 


LEVEL  OF  SIGNIFICANCE  ~ & 
(CRITICAL  VALUE  .05) 


HqI  Th  .re  is  no  agreement  among  those  interviewed  regarding  the 
difficulty  of  performing  the  assessment  task. 

There  tends  to  be  agreement  among  those  interviewed  regarding 
the  difficulty  of  performing  the  assessment  task. 


Figure  9.  Results  of  Coefficient  of  Concordance  Tests — 

Difficulty  of  Assessment  Tasks  (Questions  15a  & 16a) 
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Difference  from  the  task  ranked  most  difficult 
is  marginally  significant  (.05  < a < .1). 
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Problem  Areas.  The  responses  given  for  the  task  than  tended 
to  be  ranked  higher  in  difficulty  do  not  indicate  that  the  personnel  inter- 


viewed perceived  that  the  accomplishment  of  these  tasks  is  a significant 
problem  area.  Four  laboratory  personnel  indicated  that  obtaining  infor- 


mation regarding  the  priority  of  a program  was  "virtually  impossible,"  but 
the  mean  response  (see  Table  F-VI)  was  3.00.  This  suggests  that,  overall, 
these  personnel  perceived  that  obtaining  information  regarding  priority 
was  "somewhat  difficult."  Similar  results  are  noted  when  the  task  of 
obtaining  information  regarding  "ADTP  cost  estimates"  is  considered.  The 
mean  of  the  responses  given  for  this  task  is  3.65,  and  thus  also  fell 
within  the  "somewhat  difficult"  classification.  The  mean  of  the  response 
given  by  the  product  division  personnel  regarding  the  task  of  assessing 
"ADTP  cost  estimates"  also  fell  within  the  "somewhat  difficult"  category 
(3.24;  see  Table  F-VII). 

The  above  conclusion  is  corroborated  by  the  fact  that  five  of  the 
six  demographic  groupings  considered  failed  to  exhibit  agreement  regarding 
the  relative  ease  or  difficulty  of  accomplishing  tne  assessment  tasks.  It 
is  felt  that  if  a significant  problem  existed  in  accomplishing  the  assess- 
ment tasks,  then  agreement  would  have  been  found  as  a result  of  these  tests. 
Product  Division  Time  and  Manpower  (Questions  15b  and  16b) 

General.  Questions  15b  and  16b  were  both  stated  as  follows: 

Questions  15b  and  16b:  Do  you  think  that  the  product 

divisions  have  enough  time  and  manpower  to  devote  to 
the  assessment  process? 

These  two  questions  were  formulated  because  of  recommendations  received 
during  trial  interviews.  Therefore,  only  the  thirty-four  personnel  inter- 
viewed after  the  trial  interviews  were  asked  these  questions.  Two  answer 
choices  were  provided — "yes"  and  "no." 
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These  questions  do  not  address  the  issue  of  product  division 
time  and  manpower  availability  from  an  organization-specific  perspective. 
Thus,  the  nonorganizational  demographic  groupings  were  considered  in  the 
analysis.  The  same  analytical  technique,  used  to  analyze  Question  14  re- 
sponses were  used  to  analyze  the  responses  given  in  answer  tc  these 
questions. 

Findings.  The  responses  given  in  answer  to  these  questions 
suggest  that  the  personnel  interviewed  perceived  that  the  product  divi- 
sion will  have  enough  time  and  manpower  to  devote  to  the  assessment  process. 
This  finding  is  based  upon  a consideration  of  the  95%  confidence  intervals 
computed  for  the  percentage  of  personnel  who  will  answer  "yes"  to  these 
questions.  These  confidence  intervals  are  shown  in  Figure  10.  This 
figure  is  constructed  in  the  same  manner  of  Figure  8,  and  it  can  be  seen 
from  the  results  shown  that  the  confidence  intervals  fall  decidedly  above 
the  median  value  of  50%  for  the  All  Personnel  and  GS-15  Lt  Col  and  Above 
demographic  groupings.  Also,  the  lower  bound  of  the  confidence  interval 
falls  near  the  value  of  35%  for  six  of  the  demographic  groups,  and  the 
corresponding  value  of  the  upper  bound  of  the  confidence  interval  falls 
near  the  value  of  85%. 

The  subjective  comments  received  as  a result  of  these  questions 
suggest  that  the  product  division  personnel  think  that  performing  the  as- 
sessment is  an  important  function  and  that  time  should  be  made  available 
to  accomplish  the  task.  These  comments  also  suggest  that  some  personnel 
feel  that  they  cannot  devote  as  much  time  to  the  assessments  as  they 
would  prefer.  The  following  excerpts  illustrate  their  views* 

1.  Time  has  to  be  made  available.  We  can't  afford 
not  to  do  the  assessments  properly. 
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2.  I think  that  the  assessment  process  is  valu- 
able. We  should  do  it  in  preference  to  some  of  the 
"mickey  mouse"  exercises  we  have  to  do  now.  We  do 
have  deadlines,  and  we  really  can't  do  as  good  a job 
as  you  ought  to  be  allowed. 

3.  You  can  never  get  a handle  on  the  assessment 
in  the  way  they  would  like  for  you  to  do  it. 

4.  We  could  have  spent  more  time  on  "payoff." 

To  really  assess  payoff  you  have  to  do  a detailed 
system  analysis.  . . . 

The  confidence  intervals  computed  for  the  demographic  groupings 
based  upon  a person's  grade  depart  from  the  generalization  stated  above. 

The  results  shown  in  Figure  10  suggest  that  a larger  percentage  of  the 
higher  grade  personnel  would  be  expected  to  answer  "yes"  than  the  lower 
grade  personnel.  Moreover,  the  difference  in  the  responses  given  by  the 
two  groups  was  found  to  be  significant  at  the  .056  level.  Thus,  the 
data  collected  suggest  that  the  higher  grade  personnel  tend  to  feel  that 
enough  time  and  manpower  does  exist;  whereas  the  lower  grade  personnel 
tend  to  be  more  pessimistic  regarding  this  issue. 

Recommendation.  It  is  recommended  that  the  concerns  expressed 
by  the  personnel  in  the  lower  grades  be  recognized  by  the  laboratory  and 
product  division  management  personnel.  The  planning  activities  regarding 
new  ADTPs  should  include  sufficient  time  for  the  product  divisions  to  com- 
plete the  assessments,  and  the  supervisory  personnel  within  the  product 
divisions  should  establish  appropriate  in-house  priority  for  accomplishing 
the  at  .essments. 

Difference  Between  Demographic  Groups.  The  statistical 
hypothesis  that  there  tends  to  be  no  difference  in  the  responses  given 
by  the  personnel  in  the  different  demographic  groupings  was  tested  using 
the  contingency  table  test.  No  instance  was  found  for  which  the  hypothesis 
could  be  rejected  at  a critical  value  of  .05. 
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Laboratory  Responsiveness  (Question  17) 

General.  The  personnel  were  asked  to  give  their  perception  of 

the  degree  of  laboratory  responsiveness  in  answer  to  interview  Question  17. 

Question  17:  Another  goal  of  the  ADTP  assessment  policy 

is  to  increase  the  responsiveness  of  the  laboratories 
to  the  product  divisions  with  respect  to  the  conduct 
of  advanced  development  programs.  Please  indicate  your 
thought  on  the  degree  of  responsiveness  that  should 
exist  and  what  will  exist  now  that  the  ADTP  assessment 
policy  is  in  effect. 

All  the  personnel  interviewed  answered  this  question.  Also,  note  that  the 
question  required  those  interviewed  to  consider  two  different  situations — 
the  degree  of  responsiveness  that  should  exist  and  the  degree  of  respon- 
siveness that  will  exist.  The  two  responses  given  by  each  person  inter- 
viewed were  considered  jointly  as  a paired  response,  and  the  sign  test 
was  used  to  analyze  the  data. 

The  null  hypothesis  that  now  the  ADTP  assessment  policy  is  in  ef- 
fect the  laboratories  will  tend  to  be  more  responsive  to  the  product  divi- 
sions than  they  should  be  was  tested.  Each  of  the  thirteen  demographic 
groups  defined  in  Table  V were  considered  in  the  analysis.  The  results 
of  the  hypothesis  test  performed  are  shown  in  Table  XXI  and  Figure  11. 

Table  XXI  is  constructed  in  order  to  illustrate  the  responses 
given  by  the  different  demographic  groupings.  The  first  ta^le  -olumn 
lists  the  short  title  of  the  demographic  grouping.  The  values  shown  in 
the  next  three  columns  represent  the  number  of  persoa~''l  who  answered 
this  question  such  that  their  responses  imply  the  perception  identified 
by  the  table  column  heading.  The  entries  shown  in  the  last  table  column 
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values  are  also  plotted  in  bar  chart  form  in  Figure  11.  This  figure  is 
constructed  in  the  same  manner  as  Figure  5. 
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DEMOGRAPHIC  GROUPING 

All  Personnel 
n * 38 

All  Laboratories 
n = 17 

Product  Division  (All 
Organizations)  n = 17 

Laboratory  A Only 
n = 6 

Laboratory  B Only 
n = 9 

Product  Division  (Org  A 
Only)  n = 8 

Product  Division  (Org  B 
Only)  n * 9 

GS-15/Lt  Col  and  Above 
n = 13 

GS-14/Maj  and  Below 
n = 21 

Over  20  Yrs  Experience  in 
R&D  Field  n = 19 

Under  20  Yrs  Experience  in 
R&D  Field  n = 15 

More  Than  2 Yrs  in  Present 
Job  Assignment  n = 21 

Less  Than  2 Yrs  in  Present 

Job  Assignment  n c 13  1 , | .1.1  . I . I ■ LJ — i — LJ — i — J 

1.0  .1  .01  .001  10  <10 

LEVEL  OF  SIGNIFICANCE  ~ & 
(CRITICAL  VALUE  .05) 

Hi  Now  that  the  ADTP  assessment  policy  is  in  effect,  the  lab- 

0 oratories  will  tend  to  be  more  responsive  to  the  product 
divisions  than  they  should  be. 

H : The  laboratories  will  not  tend  to  be  more  responsive  to  the 

1 product  divisions  than  they  should  be. 


Figure  11.  Results  of  the  Sign  Test-Question  17 
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Findings.  The  results  of  the  analysis  indicate  that  the  null 
hypothesis  can  be  rejected  for  the  All  Personnel  demographic  grouping. 
Thus,  it  can  be  concluded  that,  overall,  the  personnel  interviewed  per- 
ceived that  the  laboratories  would  not  be  more  responsive  than  they  should 
be.  Furthermore,  twenty-six  (68%)  of  the  personnel  perceived  that  the 
laboratories  will  be  less  responsive  than  they  should  be;  whereas  only 
nine  (24%)  individuals  perceived  that  the  laboratories  will  be  as  respon- 
sive as  they  should  be.  This  finding  suggests  that  those  interviewed  per- 
ceived that  the  laboratories  will  be  less  responsive  to  the  product  divi- 
sion than  they  should  be. 

The  results  shown  in  Table  XXI  and  Figure  11  indicate  that  the 
null  hypothesis  can  be  rejected  for  the  All  Product  Division  Organiza- 
tions demographic  grouping,  but  not  for  the  All  Laboratories  demographic 
grouping.  It  can  also  be  seen  that  fifteen  (88%)  of  the  product  division 
personnel  perceived  that  the  laboratories  will  be  less  responsive  than 
they  should  be.  The  laboratory  personnel  tend  to  disagree  with  this 
view;  however,  they  did  not  exhibit  a strong  consensus  of  opinion.  Three 
(18%)  individuals  indicate  that  they  felt  that  the  laboratories  will  be 
more  responsive;  five  (29%)  indicated  as  responsive;  nine  (53%)  indicated 
less  responsive. 

The  remaining  demographic  groupings  also  differed  in  the  percep- 
tion of  laboratory  responsiveness.  The  null  hypothesis  can  be  rejected 
for  the  GS-15  Lt  Col  and  Above  grouping,  but  not  for  the  GS-14  Maj  and 
Below  grouping.  Similar  results  can  be  seen  to  exist  for  the  longevity 
demographic  groupings.  Thus,  it  appears  that  a person's  grade,  length  of 
service  in  the  R&D  field,  and  time  in  present  job  assignment  may  affect 
the  perception  held  regarding  the  laboratories'  responsiveness. 
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Personnel  Attitude  (Question  18) 

General.  This  question  was  stated  as  follows: 

Question  18:  The  ADTP  assessment  policy  is  a good  idea. 

All  of  the  personnel  interviewed  answered  this  question,  and  many  offered 
additional  comments  illustrating  their  view  of  the  assessment  process. 
Findings 

General.  In  general,  the  personnel  interviewed  were  very 
favorable  to  the  assessment  process.  Twenty-seven  (71%)  individuals 
marked  either  "agree”  or  "strongly  agree  ' in  response  to  this  question, 
and  thirty-fou^  (89%)  marked  a response  from  "inclined  to  agree"  to 
"strongly  agree." 

The  responses  were  also  considered  by  demographic  groupings.  Con- 
fidence intervals  were  computed  for  the  percentage  of  personnel  expected 
to  answer  "inclined  to  agree"  or  higher.  These  confidence  intervals  are 
shown  in  Figure  12.  This  figure  is  constructed  in  the  same  manner  as 
Figure  8.  The  results  shown  in  Figure  12  indicate  a favorable  attitude 
toward  the  assessment  process.  Approximately  65%  of  the  personnel  can 
reasonably  bo  expected  to  be  in  favor  of  the  assessment  process,  even 
if  the  lower  bound  of  the  confidence  interval  is  assumed  to  represent 
the  true  value  of  the  percentage.  It  can  also  be  seen  that  very  little 
difference  exists  between  the  confidence  intervals  computed  for  the  various 
demographic  groupings. 

Difference  Between  Demographic  Groupings.  The  statistical 
hypothesis  that  there  is  no  difference  between  the  responses  given  by  in- 
dividuals in  two  related  demographic  groupings  was  tested  using  the 
Smirnov  two-sample  test  (see  Chapter  III).  This  hypothesis  could  not  be 
rejected  at  a reasonable  level  of  significance  for  any  of  the  six  pairs 


127 


GSM/SM/76S-21 


DEMOGRAPHIC  GROUPING 


95X  CONFIDENCE  INTERVALS 


All  Personnel 

n = 38 


All  Laboratories 
n = 17 

Product  Divisions  (All 
Organizations)  n = 17 

GS-15/Lt  Col  and  Above 
n = 13 


GS-14/Maj  and  Below 

n = 21 


Over  20  Yrs  Experience  in 
R&D  Field  n = 19 


Under  20  Yrs  Experience  in 
R&D  Field  n = 15 


More  Than  2 Yrs  in  Present 
Job  Assignment  n = 21 

Less  Than  2 Yrs  in  Present 
J-o  Assignment  n = 13 


20  40  60  80 

CONFIDENCE  INTERVAL  ~ 1 


Question  18:  The  ADTP  assessment  policy  is  a good  idea. 


Bar  shown  represents 

95%  confidence  intervals  for  the  percentage  of  personnel  who 
will  answer  "inclined  to  agree"  to  "strongly  agree." 


Figure  12.  95 7.  Confidence  Intervals — Question  18 
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of  demographic  groupings  considered.  The  demographic  grouping  pairs 
considered  were  the  same  as  those  considered  during  the  analysis  of  the 
responses  to  Question  6. 

Personnel  with  Favorable  Attitudes.  For  the  purposes  of 
analysis,  an  individual  who  gave  a response  from  "inclined  to  agree"  to 
"strongly  agree"  was  considered  to  be  in  favor  of  the  assessment  process. 

Some  of  the  comments  offered  by  these  personnel  were  favorable,  and  some 
pointed  out  some  areas  of  concern.  The  favorable  comments  are  summarized 
below: 

1.  The  policy  forces  specific  comments  from  the  product  divi- 
sions and  creates  a corporate  spokesman. 

2.  The  policy  improves  communication  among  the  various  product 
division  organizations. 

3.  The  policy  will  increase  the  likelihood  of  an  ADTP  being  used 
by  the  product  divisions. 

The  areas  of  concern  expressed  can  be  summarized  as  follows: 

1.  The  assessment  idea  is  good  in  theory,  but  the  implementation 
is  questionable.  The  timeliness  of  the  product  division  as- 
sessments, the  bias  of  an  assessment,  and  the  lack  of  visi- 
bility as  to  how  the  assessments  will  be  used  by  HQ  AFSC  were 
cited  as  areas  of  concern  regarding  the  implementation  of 
the  assessment  process. 

2.  There  is  a danger  of  overemphasis  being  placed  on  pleasing 
the  product  divisions.  The  long  term  technology  advances 
may  suffer. 

The  individual  who  regarded  timeliness  of  the  assessment  process  as  a problem 
area  indicated  that  the  product  division  had  been  working  on  the  assessment 
he  was  familiar  with  znv  "almost  a year."  The  individual  who  indicated 
that  he  was  not  aware  of  how  HQ  AFSC  uses  the  assessments  was  one  of  the 
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made  it  a practice  to  ask  the  remaining  personnel  interviewed  if  they  were 
aware  of  how  the  assessments  were  used  or  if  they  had  received  feedback 
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from  HQ  AFSC  regarding  the  assessments  they  were  familiar  with.  None  of 
these  individuals  indicated  a positive  answer  to  this  verbal  question. 

Recommendations.  Two  recommendations  are  offered  regarding 
the  implementation  of  the  assessment  process.  First,  it  is  recommended 
that  a definite  time  period  be  established  for  the  product  division  to 
complete  an  assessment  in  the  event  that  the  forty-five  day  time  period 
defined  in  the  implementing  regulation  cannot  be  met.  It  is  recognized 
that  the  forty-five  day  period  may  be  insufficient  to  conduct  appropriate 
research  and  negotiation  regarding  a complex  ADTP,  but  it  is  felt  that 
the  assessment  period  should  not  be  left  open  ended  in  these  situations. 
Secondly,  it  is  recommended  that  consideration  be  given  to  improving  the 
downward  communication  regarding  the  use  of  the  assessments  by  HQ  AFSC 
and  the  adequacy  of  the  assessments  themselves. 

Personnel  with  Unfavorable  Attitude.  Personnel  were  con- 
sidered to  hold  an  unfavorable  attitude  regarding  the  assessment  process 
if  they  indicated  a response  of  "strongly  disagree"  to  "undecided."  One 
laboratory  individual  marked  "strongly  disagree"  in  answer  to  this  ques- 
tion. He  offered  the  following  comment. 

You  don't  sell  a program  by  writing  a plan.  First 
you  sell  it;  then  you  write  the  plan.  I think  the  lab 
should  work  closely  with  the  product  divisions  in  the 
planning  of  an  ADTP,  and  I think  the  product  divisions 
should  have  a vote  regarding  an  ADTP,  but  I don’t  think 
they  should  have  a veto.  The  assessment  policy  gives 
thorn  a veto.  The  assessments  have  been  too  short-term; 

I think  the  process  will  be  eliminated.  Air  Staff  dis- 
agrees with  XR  having  that  much  power.  They  have  al- 
ready turned  around  one  assessment. 

Also,  one  product  division  individual  marked  "disagree"  In  answer  to  the 
question.  The  comment  provided  by  this  individual  is  stated  below. 

We  have  super  internal  coordination  problems.  I 
rewrote  my  assessment  at  least  six  times.  Both  groups 
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have  to  take  responsibility.  The  product  divisions  have 
the  authority  to  kill  a program  now.  Khat  happens  if 
a good  program  is  cancelled  because  of  a negative  prod- 
uct division  assessment? 

It  can  be  seen  from  these  two  comments  that  these  personnel  expressed 
concern  regarding  an  apparent  shift  of  ADTP  decision  power  to  the  product 
division. 

Two  individuals  indicated  that  they  were  undecided  regarding  the 
assessment  process.  Only  one  of  them  commented  on  his  answer.  He  ex- 
pressed the  following  view: 

I think  it  is  just  a way  to  make  money  decisions  for 
people  at  AFSC,  They  are  getting  somebody  to  do  their 
job  for  them. 

Additional  Technology  Transfer  Mechanisms  (Question  19) 

General.  The  personnel  interviewed  were  given  a description  of 
five  additional  actions  that  could  be  taken  to  improve  the  transfer  of 
technology  and  were  asked  if  these  actions  had  enough  merit  to  warrant 
further  consideration.  The  specific  actions  proposed  are  described  in 
Chapter  III.  under  the  section  entitled  "Additional  Technology  Transfer 
Mechanisms."  All  of  the  personnel  interviewed  answered  this  question,  a 
and  the  mean  response  given  by  the  various  demographic  groupings  varied 
from  2.11  to  5.11  (see  Table  F-VIII).  This  numerical  range  corresponds  to 
a word  answer  range  of  "disagree"  to  "inclined  to  agree."  Thus,  the  mean 
responses  tend  toward  the  "disagree"  end  of  the  measurement  scale.  This 
suggests  that,  in  general,  those  interviewed  did  not  perceive  that  the 
proposed  actions  warranted  further  consideration.  However,  the  analysis 
performed  indicates  that  some  of  these  actions  are  less  preferred  than  others. 
Perception  of  Proposed  Mechanisms 

Agreement.  The  null  hypothesis  of  no  agreement  was  tested 
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using  the  coefficient  of  concordance  test.  Each  of  the  thirteen  demo- 
graphic groupings  was  considered,  and  the  results  of  these  tests  are 
shown  in  Figure  13.  This  figure  is  constructed  in  the  same  manner  as 
Figure  5. 

It  can  be  seen  that  when  the  responses  from  all  of  the  personnel 
interviewed  were  considered  as  a group,  significant  agreement  was  found 
to  exist.  However,  when  the  responses  were  considered  by  demographic 
groupings,  four  of  the  twelve  remaining  groupings  failed  to  exhibit  sig- 
nificant agreement.  Not?  that  the  agreement  for  the  All  Laboratories 
grouping  is  considered  significant  (a  = .052)  although  the  value  of  the 
level  of  significance  is  slightly  greater  than  the  critical  value  of  .05. 

It  was  postulated  that  the  lack  of  agreement  observed  for  the 
Under  20  Years  and  Less  Than  2 Years  demographic  groupings  might  be  caused 
by  the  mixture  of  laboratory  and  product  division  personnel  in  these  two 
groupings.  Accordingly,  the  no-agreement  null  hypothesis  was  again  tested, 
and  in  these  tests  the  responses  given  by  the  laboratory  and  product  divi- 
sion personnel  which  comprised  these  two  groups  were  considered  separately. 
The  smallest  value  of  the  level  of  significance  obtained  (a  = .18)  would 
not  allow  for  the  rejection  of  the  null  hypothesis.  Thus,  it  is  concluded 
that  the  differences  in  agreement  shown  in  Figure  13  for  these  longevity 
demographic  groupings  are  in  fact  due  to  longevity  factors  and  not  to  the 
mixture  of  laboratory  and  product  division  personnel. 

Preferred  Order.  Estimates  of  the  true  preferential  order 
of  the  additional  technology  transfer  mechanisms  are  shown  in  Table  XXII. 
This  table  is  constructed  in  the  same  manner  as  Table  VIII.  The  results 
shown  in  Table  XXII  indicate  that  the  mechanism  "make  future  funding  con- 
tingent on  past  transfers  was  unanimously  the  least  preferred  action. 


DEMOGRAPHIC  GROUPING 

All  Personnel 
n e 38 

All  Laboratories 
n = 17 

Product  Division  (All 
• Organizations)  n = 17 

Laboratory  A Only 
n = 6 

Laboratory  B Only 
n = 9 

Product  Division  (Org  A 
Only)  n = 8 

Product  Division  (Org  B 
Only)  n =*  9 

GS-15/Lt  Col  and  Above 
n = 13 

GS-14/Maj  and  Below 
n = 21 

Over  20  Yrs  Experience  in 
R&D  Field  n = 19 

Under  20  Yrs  Experience  in 
R&D  Field  n = 15 

More  Than  2 Yrs  in  Present 
Job  Assignment  n = 21 

Less  Than  2 Yrs  in  Present 
Job  Assignment  n = 13 


LEVEL  OF  SIGNIFICANCE  ~ & 
(CRITICAL  VALUE  .05) 


HqX  There  is  no  agreement  among  those  interviewed  regarding  which 
additional  technology  transfer  mechanism  that  should  be  given 
further  consideration. 


H^s  There  tends  to  be  agreement  among  those  interviewed  regarding 
further  consideration  of  these  mechanisms. 
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Furthermore,  the  personnel  in  three  of  the  demographic  groupings  assigned 
ranks  so  that  the  difference  between  the  rank  assigned  to  this  mechanism 
and  that  assigned  to  the  highest  ranked  mechanism  is  significant  at  the  .05 
level,  and  three  assigned  ranks  so  that  the  difference  is  significant  at 
the  .1  level. 

Differences  between  the  ranks  assigned  to  the  other  proposed 
mechanisms  were  not  found  to  be  significant  at  a reasonable  level  of  sig- 
nificance. However,  it  can  be  seen  from  the  results  shown  in  Table  XXII 
that  the  data  suggest  that  the  rating  schemes  tend  to  be  preferred  over 
establishing  a technology  transfer  agency.  It  should  be  noted  that  this 
conclusion  does  not  imply  that  the  rating  schemes  were  endorsed.  If  the 
mean  of  the  responses  assigned  to  the  highest  ranked  mechanism,  "product 
divisions  rate  laboratories,"  are  considered  (see  Table  F-VIII),  it  can 
be  seen  that  these  mean  values  range  from  3.67  to  5.11.  Thus,  it  can  be 
concluded  that  the  additional  technology  transfer  mechanisms  proposed 
were  not  strongly  endorsed  by  the  personnel  interviewed. 

Several  personnel  offered  comments  regarding  these  proposed  mech- 
anisms. The  general  nature  of  hese  comments  are  summarized  below. 


1.  The  mechanisms  place  too  much  emphasis  on  transferring  tech- 
nology. Sometimes  it  is  best  not  to  transfer  a bad  technology. 


2,  A technology  transfer  agency  is  a good  idea  in  theory,  but 
in  practice  it  would  probably  create  more  bureaucracy  and 
be  a drain  on  manpower.  This  function  could  best  be  ac- 
complished in  the  present  development  planning  organizations. 


3.  There  are  many  factors  which  determine  whether  an  ADTP  is 
used  in  a system  application.  Therefore,  future  funding 
should  not  be  contingent  upon  past  transfers  unless  it 
is  limited  to  a basis  for  reward. 


A.  The  rating  systems  will  probably  just  lead  to  a contest. 
Differences  Between  Demographic  Groupings.  The  statistical 


L 


135 


GSM/SM/76S-21 

hypothosi  r,  that  there  tends  to  bo  no  difference  in  the  responses  given  by 
the  personnel  in  the  various  demographic  groupings  was  tested.  The  pro- 
cedure followed  was  the  same  as  described  for  this  test  in  Question  6, 

One  instance  was  found  in  which  the  difference  was  significant  at  the  .05 
level.  It  was  determined  that  the  laboratory  and  product  division  per- 
sonnel tend  to  respond  differently  in  scoring  the  mechanism  "make  future 
funding  contingent  upon  past  transfers."  The  mean  of  the  responses  given 
by  laboratory  personnel  was  found  to  be  2.59.  The  corresponding  value 
for  the  product  division  personnel  was  3.24.  Thus,  both  groups  of  indi- 
viduals tend  to  reject  chis  concept;  the  significant  difference  found  was 
due  to  the  fact  that  the  laboratory  personnel  tend  to  reject  the  concept 
more  strongly  than  the  product  division  personnel. 


Chapter  Summary 

A detailed  analysis  of  the  data  collected  during  the  structured 
interviews  is  presented  in  this  chapter.  Generally,  the  discussion  pro- 
ceeds on  a question  by  question  basis.  The  results  of  the  analysis  are 
synthesized  into  conclusions  and  recommendations  in  the  next  chapter. 
Chapter  V.  A summary  of  these  results  in  therefore  not  repeated  in  this 


chapter  summary. 
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V.  Conclusions  anti  Recommendations 

The  analysis  performed,  in  Chapter  IV  was  structured  around,  the 
interview  questions.  It  is  felt  that  a question  by  question  formulation 
of  conclusions  and  recommendations  would  be  fragmented.  Accordingly  the 
principal  conclusions  and  recommendations  formulated  as  a result  of  this 
research  effort  are  presented  in  relation  to  the  objectives  of  the  research 
effort  stated  in  Chapter  I.  In  summary  form,  these  objectives  are  as  fol- 
lows* (1)  characterize  the  perceived  effectiveness  of  the  ADTP  assessment 
policy,  (2)  determine  if  this  perceived  effectiveness  is  dependent  upon 
various  demographic  groupings,  (3)  identify  problem  areas  and  recommend 
solutions,  (4)  determine  the  general  attitude  of  personnel  with  respect 
to  the  as.'  ssn.jnt  process,  and  (5)  explore  the  desirability  of  introducing 
additional  techno.’ ^gy  transfer  mechanisms. 

Perceived  Effectiveness  of  ADTP  Assessments 

Overall,  the  personnel  interviewed  perceived  that  the  assessment 
process  will  foster  a relationship  between  the  laboratories  and  the  prod- 
uct division  which  will,  in  general,  be  conducive  to  the  transfer  of  tech- 
nology to  system  application.  Those  interviewed  also  perceived  that  this 
relationship  would  tend  to  encourage  improved  communication  and  dialogue 
between  the  laboratory  and  product  division  personnel.  Moreover,  it  ran 
be  concluded  that  the  personnel  interviewed  perceived  that  the  ADTP  assess- 
ment policy  would  not  tend  to  make  the  laboratories  more  responsive  than 
they  should  be.  However,  some  laboratory  personnel  were  found  to  disagree 
with  this  conclusion. 

Transfer  of  Technology.  The  personnel  interviewed  perceived  that 
the  assessment  policy  will  foster  a relationship  which  will  enhance  the 
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transfer  of  technology  in  two  ways.  First,  the  data  collected  suggest 
that  a balance  between  the  push  of  technology  and  the  pull  of  the  product 
division  need  is  likely  to  be  enhanced  by  the  assessment  process.  This 
conclusion  is  derived  from  the  following  specific  findings  and  observa- 
tions (referenced  to  appropriate  figures  and  tables  in  Chapter  IV). 

1.  There  tended  to  be  agreement  among  those  interviewed  regarding 
which  technology  transfer  aiding  events  the  assessment  policy 
will  encourage  (see  Figure  5). 

2.  The  events  "establish  an  early  informational  linkage"  and 
create  "strong  product  division  support"  were  perceived  to 
be  those  most  likely  to  be  encouraged  by  the  assessment 
process  (see  Table  VIII). 

3.  The  creation  of  an  early  informational  linkage  and  a strong 
manufacturer  (i.e,,  product  division)  support  have  been  cited 
by  other  researchers  as  two  principal  factors  required  in 
order  to  establish  an  equilibrium  between  technology  push 
and  demand  pull  (Teich,  1974 «42,  43;  ODDR&E,  1969i47). 

4.  "Establish  that  technology  is  mature"  and  "provide  a forum 
for  problem  resolution"  were  perceived  to  be  the  two  event, 
least  likely  to  be  encouraged  by  the  assessment  process 
(see  Table  VIII). 

The  seconu  way  in  which  the  transfer  of  technology  will  likely 
be  enhanced  is  by  reducing  the  significance  of  two  barriers  to  the  trans- 
fer of  technology — "communication  barriers"  and  "lack  of  transfer  manage- 
ment structure."  This  conclusion  is  derived  from  the  following  specific 
findings! 

1.  The  proposed  barriers  to  technology  were  not  perceived  to  be 
equally  significant  (see  Chapter  IV,  Questions  7 and  8). 

2.  There  tended  to  be  agreement  among  those  interviewed  regarding 
the  significance  of  the  proposed  barriers  (see  Figure  6). 

3.  Those  interviewed  perceived  that  the  significance  of  "com- 
munication barriers"  and  "transfer  management  structure 
would  be  affected  by  the  assessment  process  (see  Tables  IX, 

X,  and  XI). 

4.  The  significance  of  the  remaining  proposed  barriers  to  tech- 
nology transfer  was  not  perceived  to  b affected  by  the  as- 
sessment process. 
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The  fourth  finding  cited  above  indicates  that  the  assessment 
process  was  not  perceived  to  redi  ce  the  significance  of  "inertia  barriers," 
"time  barriers,"  "cost  barriers,"  "technology  barriers,"  and  "geographic 
distance."  Furthermore,  the  first  three  barriers  were  perceived  to  be 
more  significant  than  the  last  two  (see  Tables  IX  and  X).  The  subjective 
comments  received  during  the  interviews  suggest  that  the  higher  perceived 
significance  of  the  first  three  barriers  may  be  due  to  the  performance, 
schedule,  and  cost  tradeoffs  that  must  be  made  during  the  weapon  system 
acquisition  process.  Thus,  the  assessment  process  was  not  perceived  as 
a final  solution  to  the  technology  transfer  problem. 

Recommendations . It  is  recommended  that  research  on  the  barriers 
to  the  transfer  of  technology  be  continued.  It  might  be  possible  to 
achieve  a better  understanding  of  the  characteristics  of  the  more  signifi- 
cant barriers  remaining  by  focusing  a thesis  effort  on  these  factors. 

Communication  and  Dialogue.  The  effect  of  the  assessment  process 
on  communication  and  dialogue  was  considered  separately.  One  conclusion 
reached  is  that  the  personnel  interviewed  did  not  perceive  that  the  visi- 
bility of  ADTPs  would  be  vastly  improved  by  the  assessment  process.  How- 
ever, it  is  felt  that  this  conclusion  may  be  biased  because  of  the  timing 
of  the  implementation  of  the  assessment  policy.  These  conclusions  were 
derived  from  the  following  specific  findings  and  observations! 

1.  The  percentage  of  the  personnel  interviewed  who  indicated 
that  they  had  not  learned  of  ADTPs  because  of  the  assess- 
ment process  was  generally  relatively  large — 37%  for  Ques- 
tion 9 and  61%  for  Question  10. 

2.  The  percentage  of  personnel  interviewed  who  indicated  that 
thev  had  learned  of  more  than  four  ADTPs  because  of  the 
assessment  process  was  relatively  small — 13%  for  Question  9 
and  10%  for  Question  10. 

3.  Many  of  the  ADTPs  were  ongoing  when  the  assessment  policy 


139 


^ ~juasi\^wst 


GSM/SM/76S-21 


was  implemented;  thus,  the  personnel  had  an  opportunity  to 
learn  to  these  programs  by  other  means  before  the  assessment 
policy  was  established. 

It  was  also  concluded  that  the  product  division  personnel  perceived 
that  communication  with  laboratory  personnel  would  be  enhanced  by  the  assess- 
ment process  and  vice  versa.  However,  the  overall  communication  pattern 


was  not  perceived  to  be  significantly  changed  by  the  assessment  process. 
These  conclusions  are  derived  from  the  following  specific  findings  (refer- 
enced to  appropriate  figures  and  tables  in  Chapter  IV): 

1.  There  tended  to  be  agreement  among  those  interviewed  regarding 
the  relative  amount  of  their  working  time  spent  in  communica- 
tion about  new  ADTPs  with  personnel  in  "higher  headquarters," 
"product  divisions,"  "laboratories,"  "AFLC,"  and  "operational 
commands"  (see  Figure  7). 

2.  Those  interviewed  perceived  that  more  of  their  working  time 
would  be  spent  in  communication  about  new  ADTPs  with  personnel 
in  "laboratories,"  "higher  headquarters,"  and  "product  divi- 
sions" than  would  be  spent  in  communication  with  personnel 

in  "operational  commands"  and  "AFLC"  (see  Tables  XIII  and  XIV). 

3.  Thirteen  (76 %)  of  the  product  division  personnel  indicated 
that  they  perceived  that  more  of  their  working  time  would 
be  spent  in  communication  with  laboratory  personnel  now 
that  the  assessment  policy  is  in  effect.  Moreover,  twelve 
(75%)  of  the  laboratory  personnel  indicated  that  they  per- 
ceived that  more  of  their  working  time  would  be  spent  in 
communication  with  product  division  personnel.  These  per- 
centages were  found  to  be  statistically  significant  (see 
Tables  XVI  and  XVII). 

A.  Neither  the  laboratory  personnel  nor  the  product  division 
personnel  indicated  that  they  perceived  that  the  amount  of 
their  working  time  spent  in  communication  with  personnel  ir. 
the  other  agencies  considered  would  be  significantly  affected 
(see  Tables  XVI  and  XVII), 

The  data  collected  tends  to  suggest  that  the  laboratory  and  product 
division  personnel  interviewed  perceived  that  more  time  should  be  spent 
in  communication  with  personnel  in  the  counterpart  agencies  than  will  be. 
However,  the  results  were  not  statistically  significant.  The  specific 
findings  are  as  follows: 
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1.  Ten  (63%)  of  the  laboratory  personnel  perceived  that  more 
time  should  be  spent  in  communication  with  product  division 
personnel  than  will  be,  and  four  (25%)  perceived  that  less 
time  should  be  spent  (see  Table  XIX,  Chapter  IV). 


2.  Ten  (59%)  of  the  product  division  personnel  perceived  that 
more  time  should  be  spent  in  communication  with  laboratory 
personnel  than  will  be,  and  two  (12%)  perceived  that  less 
time  should  be  spent  (see  Table  XIX,  Chapter  IV). 


Overall,  the  personnel  perceived  that  the  assessment  policy  will 


( » 


tend  to  improve  dialogue  between  themselves  and  personnel  in  the  counter- 
part agencies.  However,  the  strength  of  this  perception  appears  to  be 
dependent  upon  the  organization  considered.  Also,  those  personnel  who 
answered  negatively  regarding  the  dialogue  issue  did  not  express  a con- 
sensus of  opinion.  These  conclusions  are  derived  from  a consideration 
of  the  responses  given  in  answer  to  Question  14.  The  following  specific 
findings  were  detected! 

1.  Thirty-one  (84%)  of  the  personnel  interviewed  indicated  that 
the  assessment  process  would  improve  dialogue  (see  Question  14 
discussion.  Chapter  IV). 

2.  All  product  division  personnel  interviewed  perceived  that  dia- 
logue would  be  improved. 

3.  Only  personnel  assigned  to  a laboratory  perceived  that  dia- 
logue would  not  be  improved.  Six  (38%)  individuals  answered 
in  this  manner.  The  responses  given  by  laboratory  and  prod- 
uct division  personnel  was  found  to  differ  statistically. 

4.  The  subjective  comments  expressed  by  those  personnel  who  an- 
swered negatively  expressed  three  different  views  regarding 
the  improvement  of  dialogue!  (1)  Existing  channels  for  dia- 
logue, which  bypass  the  product  division  development  planning 
offices,  are  adequate.  (2)  Laboratory  personnel  will  become 
more  defensive  regarding  their  programs.  (3)  The  concept  of 
ADTP  assessments  is  not  desirable. 


C 


Laboratory  Responsiveness.  The  conclusion  that  those  interviewed 
perceived  that  the  assessment  process  would  not  result  in  the  laboratories 
being  more  responsive  than  they  should  be  in  based  on  the  responses  given 
in  answer  to  Question  17.  Some  laboratory  personnel  do  not  agree  with  this 


141 


5 wsmw' 


-i~  ?~'t  -vj  * - V-^iF  sX't 


GSM/SM/76S-21 

conclusion;  however,  these  personnel  did  not  exhibit  a strong  consensus 
of  opinion  as  to  whether  the  laboratories  will  be  either  more  responsive 
or  as  responsive  as  they  should  be.  The  following  specific  findings  sup- 
port the  above  conclusions. 


I 

t 


i 

! 


1.  The  hypothesis  that  the  laboratories  will  tend  to  be  more  re- 
sponsive than  they  should  be  could  be  rejected  when  the  re- 
sponses from  all  of  the  personnel  interviewed  were  considered 
together  (see  Table  XXI,  Chapter  IV). 

2.  The  preceding  hypothesis  could  also  be  rejected  when  the  re- 
sponses from  just  product  division  personnel  were  considered, 
but  this  hypothesis  could  not  be  rejected  when  the  responses 
received  from  laboratory  personnel  were  considered  separately 
(see  Table  XXI,  Chapter  IV). 

3.  Three  (16%)  of  the  laboratory  personnel  indicated  that  they 
felt  the  assessment  process  would  result  in  the  laboratories 
becoming  more  responsive  to  the  product  divisions  than  they 
should  be.  Five  (29%)  individuals  indicated  that  the  lab- 
oratories would  be  as  responsive  as  they  should  be,  and 
nine  (58%)  people  indicated  that  the  laboratories  would  be 

less  responsive  than  they  should  be  (see  Table  XXI,  Chapter  IV). 


Effect  of  Demographic  Factors 

Thirteen  different  organizational  and  nonorganizational  demographic 
groupii.go  were  defined  (see  Table  V)  in  order  to  investigate  the  effect 
of  demographic  factors.  The  specific  procedure  followed  in  performing 
the  analysis  are  discussed  in  Chapter  IV  under  the  section  entitled  "Tech- 
nology Transfer  Aiding  Events  (Question  6)." 

Although  exceptions  were  found,  it  is  concluded  that,  in  general, 
the  responses  obtained  in  answer  to  the  question  were  not  found  to  differ 
significantly  due  to  demographic  factors  considered.  However,  it  should 
be  noted  that  a relatively  small  number  of  interviews  were  conducted,  and 
this  resulted  in  some  of  the  demographic  groupings  being  very  small  (see 
Table  VI,  Chapter  IV).  The  small  size  of  the  sample  somewhat  limits  the 
ability  to  detect  small  differences  between  responses.  The  conclusion 
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regarding  the  effect  of  demographic  factors  is  based  upon  the  following 
observations  and  findings. 

1.  The  responses  given  by  the  personnel  were  categorized  into 
six  pairs  of  demographic  groupings.  The  answers  to  forty- 
one  different  questions  were  considered.  In  this  count  each 
answer  of  a multiple  answer  question  is  considered  as  one 
answer.  Thus,  246  different  tests  were  performed. 

2.  Only  nine  instances  were  found  in  which  the  responses  given 
by  personnel  in  different  demographic  groupings  differed  sta- 
tistically. 

The  demographic  factor  which  had  the  most  effact  on  the  responses 
were  the  individual's  organization — primarily  the  laboratory  and  product 
division  organizations.  Answers  to  the  communication  and  dialogue  ques- 
tions tend  to  be  those  most  influenced  by  demographic  factors.  The  spe- 
cific findings  supporting  these  conclusions  are  listed  below* 

1.  In  eight  of  the  nine  instances  identified  above,  the  two  demo- 
graphic groupings  were  product  division  personnel  and  labora- 
tory personnel. 

2.  In  six  of  the  above  eight  instances,  the  significant  differ- 
ence in  response  occurred  in  the  answers  to  the  communica- 
tion and  dialogue  questions  (Questions  11,  12,  and  14). 

The  remaining  instances  for  which  a significant  difference  in  the 
responses  was  found  to  exist  uo  not  appear  to  be  related  to  a central  issue. 
The  differences  are  summarized  below* 

1.  The  product  division  personnel  tended  to  indicate  that  before 
ADTP  assessments  the  "lack  of  incentive  structure"  was  a more 
significant  barrier  to  the  transfer  of  technology  than  lab- 
oratory personnel  did  (see  Table  XII,  Chapter  IV). 

2.  Laboratory  B personnel  tended  to  indicate  that  before  ADTP 
assessments  a lack  of  a "transfer  management  structure"  was 
a more  significant  barrier  to  the  transfer  of  technology 
than  laboratory  A personnel  did  (see  Table  XII,  Chapter  IV). 

3.  Although  both  laboratory  and  product  division  personnel 
tended  not  to  favor  making  funding  of  ADTP  programs  con- 
tingent upon  past  transfers,  the  laboratory  personnel 
were  found  to  be  more  strongly  opposed  to  this  idea  than 
the  product  division  personnel. 
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Problem  Areas 

Significant  Problem  Area.  It  is  concluded  that  no  significant 
problem  area  associated  with  accomplishing  the  assessment  tasks  exists. 

This  conclusion  was  derived  from  the  following  findings: 

1.  The  personnel  in  four  of  the  demographic  groupings  considered 
were  not  found  to  exhibit  agreement  regarding  the  relative 
ease  or  difficulty  of  performing  the  assessment  tasks  (see 
Figure  9,  Chapter  IV).  Thus,  the  accomplishment  of  any  one 
task  was  not  perceived  to  be  consistently  more  difficult  to 
accomplish  by  these  personnel. 

2.  The  mean  of  the  response  given  for  the  tasks  perceived  to  be 
the  most  difficult  fell  within  the  somewhat  difficult  range 
on  the  answer  scale  (see  Table  F-VI,  Appendix  F). 

Potential  Problem  Areas 

Assessments  by  One  Person.  Three  product  division  personnel 
interviewed  raised  the  issue  that,  within  a given  office,  the  assessment 
might  be  accomplished  by  only  one  person.  Furthermore,  it  was  pointed  out 
that  an  annual  reassessment  might  be  accomplished  by  an  individual  who  ' 

did  not  perform  the  original  assessment  and  that  the  new  assessor  might 
or  might  not  be  familiar  with  the  conclusion  reached  during  the  first  as- 
sessment. The  occurrence  of  these  events  might  lead  to  less  visibility  of 
laboratory  programs  within  the  product  division  and  inconsistent  assessments. 

Recommendations . It  is  recommended  thar  the  possibility  of 
the  above  situations  be  recognized  and,  if  required,  that  action  be  taken 
to  minimize  the  negative  aspects  of  one-person-assessments  within  an 
office.  One  action  that  could  be  taken  is  to  insur.  that  the  results  of 
the  assessments  are  disseminated  among  the  office  personnel.  A' sc,  an 
office  case  file  could  be  established  for  each  laboratory  pregram  assessed. 

These  files  would  provide  a record  of  all  past  comments  and  data  considered 
and  could  be  referenced  when  performing  future 
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Time  and  Manpower . Overall,  the  personnel  interviewed  per- 
ceived that  the  product  divisions  will  have  enough  time  and  manpower  to 
devote  to  the  assessment  process.  However,  some  individuals,  particularly 
those  in  the  lower  grade  demographic  groupings,  expressed  concern  regarding 
the  adequacy  of  the  time  and  manpower  availability  to  perform  the  assess- 
ment tasks  (see  Figure  10,  Chapter  IV). 

Recommendation.  It  is  recommended  that  the  concerns  expressed 
by  these  personnel  be  recognized  by  the  laboratory  and  product  division 
management  personnel.  The  planning  activities  regarding  new  ADTPs  should 
include  sufficient  time  for  the  product  divisions  to  complete  the  assess- 
ments, and  the  supervisory  personnel  within  the  product  divisions  should 
establish  appropriate  in-house  priority  for  accomplishing  the  assessments. 

Timeliness  of  Assessments.  One  individual  assigned  to  a 
laboratory  indicated  that  the  product  division  had  been  working  on  the 
assessment  that  he  was  familiar  with  for  "almost  a year."  It  is  recognized 
that  the  forty-five  day  period  allowed  by  the  implementing  regulation  may 
not  allow  for  sufficient  time  to  conduct  appropriate  research  and  nego- 
tiation if  an  ADTP  is  highly  complex.  However,  it  is  felt  that  the  c 
sessment  period  should  not  lie  left  open  ended. 

Recommendation.  It  is  recommended  that  a definite  time  period 
for  the  product  division  to  complete  the  assessment  be  established  in  the 
event  that  the  forty-five  day  time  period  is  insufficient. 

Downward  Communication  Regarding  Assessments.  As  a result  of 
a comment  received  during  the  interviews,  the  researcher  asked  ten  individ- 
uals if  they  were  aware  of  how  the  assessmei  s were  used  by  HQ  AFSC  and  if 
they  had  received  feedback  from  HQ  AFSC  regarding  the  assessments  with  which 
they  were  familiar.  No  one  indicated  a positive  answer  to  these  questions. 
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Recommendation.  It  is  recommended  that  consideration  be  Riven 

° i 

* 
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to  improving  the  downward  communication  regarding  the  use  of  the  assessments 
by  HQ  AFSC  and  the  adequacy  of  the  assessments  themselves. 

Personnel  Attitude 

Although  some  of  the  personnel  interviewed  held  an  unfavorable 
attitude  regarding  the  assessment  policy,  most  of  those  interviewed  regarded 
the  process  favorably.  This  conclusion  is  based  upon  the  following  findings: 

1.  Thirty-four  (89%)  of  those  interviewed  indicated  that  they  were 
at  least  inclined  to  agree  that  the  assessment  policy  was  a 
good  idea,  and  twenty-seven  (71%)  indicated  that  they  either 
agreed  or  strongly  agreed  with  the  assessment  policy  (see 
Chapter  IV,  "Personnel  Attitude  (Question  18)  Findings"). 

2.  No  significant  difference  was  found  in  the  responses  given  by 
personnel  in  the  various  demographic  groupings. 

The  comments  offered  regarding  the  assessment  policy  are  discussed 
in  Chapter  IV  under  the  section  entitled  "Personnel  Attitude,"  Generalizing,  » 

those  personnel  in  favor  of  the  assessment  process  felt  that  a product  divi- 
sion spokesman  would  be  created,  that  communication  would  be  improved,  and 
that  the  transfer  of  technology  would  be  improved.  Some  concern  regarding 
the  implementation  of  the  assessment  process  was  expressed,  and  the  danger 
of  neglecting  the  long  term  technology  advances  because  of  overemphasis 
being  placed  on  pleasing  the  product  division  was  voiced.  Those  personnel 
opposed  to  the  assessment  process  expressed  concern  regarding  their  percep- 
tion that  the  decision  power  for  approving  an  ADTP  was  being  placed  in  the 
product  division. 


Technology 


None  of  the  additional  technology  transfer  mechanisms  proposed 
was  strongly  endorsed  by  those  interviewed.  However,  the  rating  schemes 
proposed  generally  tended  to  be  preferred  over  either  "establishing  a 
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technology  transfer  agency"  or  "make  future  funding  contingent  on  past 
transfers."  These  conclusions  are  based  upon  the  following  specific 
findings  (referenced  to  appropriate  figures  and  tables  in  Chapter  IV). 

1.  Personnel  in  nine  of  the  thirteen  demographic  groupings 
exhibited  agreement  regarding  the  ranking  of  the  proposed 
mechanism  (see  Figure  13). 

2.  The  mean  of  the  responses  given  for  the  proposed  mechanisms 
generally  fell  toward  the  "disagree"  end  of  the  measurement 
scale  (see  Table  F-VIII). 

3.  The  rating  schemes  were  consistently  ranked  the  highest  by 
the  personnel  in  the  various  demographic  groupings  (see 
Table  XXII). 

4.  The  establishment  of  a technology  transfer  agency  was  ranked 
the  second  least  preferred  by  the  personnel  in  seven  of  the 
demographic  groupings  (see  Table  XXII), 

5.  Tying  future  funding  to  past  transfers  of  technology  was 
ranked  the  least  preferred  by  the  personnel  in  all  of  the 
demographic  groupings  (see  Table  XXII), 

Chapter  Summary 

As  a summary  statement  it  can  be  concluded  that  those  interviewed 
perceived  that  the  ADTP  assessment  policy  will  tend  to  enhance  both  the 
transfer  of  technology  and  communication  and  dialogue  between  the  labor- 
atories and  product  divisions.  Also,  no  significant  problems  were  per- 
ceived to  exist  in  accomplishing  the  assessment  tasks.  Generally,  those 
interviewed  perceived  that  the  laboratories  will  not  tend  to  be  more  re- 
sponsive than  they  should  be,  but  the  laboratory  personnel  tend  to  disagree 
with  this  conclusion.  A favorable  attitude  toward  the  assessment  policy 
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were  not  strongly  endorsed. 
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Abbreviations 

ADP  - Advanced  Development  Program 
ADTC  - Armament  Development  and  Test  Center 
ADTP  - Advanced  Development  Technology  Program 
AF  - Air  Force 

AFAL  - Air  Force  Avionics  Laboratory 

AFAPL  - Air  Force  Aeropropulsion  Laboratory 

AFATL  - Air  Force  Armaments  Technology  Laboratory 

AFCRL  - Air  Force  Cambridge  Research  Laboratory 

AFFDL  - Air  Force  Flight  Dynamics  Laboratory 

AFGL  - Air  Force  Geophysics  Laboratory 

AFHRL  - Air  Force  Human  Resources  Laboratory 

AFIT  - Air  Force  Institute  of  Technology 

AFLC  - Air  Force  Logistics  Command 

AFR  - Air  Force  Regulation 

AFRPL  - Air  Force  Rocket  Propulsion  Laboratory 

AFSC  - Air  Force  Systems  Command 

AFSCP  - Air  Force  Systems  Command  Pamphlet 

AFSCR  - Air  Force  Systems  Command  Regulation 

AFWAL  - Air  Force  W”'  'it  Aeronautical  Laboratories 

AFWL  - Air  Force  Weapons  Laboratory 

ARL  - Aerospace  Research  Laboratories 

ASD  - Aeronautical  Systems  Division 
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DLSIE  - Defense  Logistics  Studies  Information  Exchange 
DOD  - Department  of  Defense 
DOL  - Director  of  Laboratories 
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DS&T  - Director  of  Science  and  Technology 
ESD  - Electronic  Systems  Division 
FCRC  - Federal  Contract  Research  Centers 
FY  - Fiscal  Year 

HQ  AFSC  - Headquarters  Air  Force  Systems  Command 

HQ  AFSC  (CV)  - Vice  Commander,  Headquarters  Air  Force  Systems  Command 

HQ  AFSC  (DL)  - Director  of  Science  and  Technology,  Headquarters  Air  Force 
Systems  Command 

HQ  AFSC  (XL)  - Development  Plans,  Headquarters  Air  Force  Systems  Command 
HQ  USAF  - Headquarters  United  States  Air  Force 

JPG  - Joint  Planning  Group  formed  at  HQ  AFSC  to  develop  a proposed  in- 
vestment strategy 

LUS  - Laboratory  Utilization  Study 
NAE  - National  Academy  of  Engineering 
NIH  - Not  invented  here 
NSF  - National  Science  Foundation 

NTIS  - National  Technical  Information  Exchange  Service 

ODDR&E  - Office  of  the  Director  of  Defense  Research  and  Engineering 

00C  - Office  of  the  Comptroller 

OSD  - Office  of  the  Secretary  of  Defense 

PMD  - Program  Management  Directive 

R&D  - Research  and  Development 

RADC  - Rome  Air  Development  Center 

RDT&E  - Research,  Development,  Test,  and  Evaluation 

RML  - Range  Measurements  Laboratory 

RTD  - Research  and  Technology  Division 

SAMSO  - Space  and  Missle  Systems  Organization 

SPG  - Systems  Program  Office 
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TPP  - Technology  Program  Plan 
USAF  - United  States  Air  Force 

6.1  - Research 

6.2  - Exploratory  Development 

6.3  - Advanced  Development 

6. A - Engineering  Development 
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DEPARTMENT  OF  THE  AIR  FORCE 

HEAOCKIARTERE  AIR  FORCE  BV»T*M»  COMMAND 
ANDREW*  AIR  FORCE  EASE,  DC  WtM 


2 9 M 

mm  * to 
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•u*ecr  Monitor  and  Control  of  Laboratory  Advanced/Engineering  Development 
Programs 

to  ADTC  ASD  ESD  SAMSO  AFAL  AFAPL  .'FAIL  AFCRL 

AFFDL  AFHRL  AFML  AFRPL  AFVIL  AMD  h*DC 


(Commander/Di rector) 

1.  The  Laboratory  Utilization  Soidy,  chaired  by  Maj  Gen  Chapman, 
recommended  that  AFSC  Laboratories'  responsiveness  to  the  Product 
Divisions  be  increased,  especially  in  the  conduct  of  Advanced  Develop- 
ment (6.3)  and  Engineering  Development  (6.4)  Programs.  The  desired 
result  is  to  Improve  the  transition  of  technology  to  system  application. 

2.  Implementation  of  this  recommendation  is  being  staffed  within  this 
Headquarters.  Toward  this  end,  it  is  envisioned  that  the  Laboratories 
will  annually  submit  6. 3/6. 4 program  documentation  to  AFSC  through  trie 
appropriate  Product  Dlvlslon(s).  The  Product  Divisions  will  add  an 
assessment  of  the  relevance,  cost  implications,  timeliness,  payoff 

and  relative  priority  of  proposed  efforts,  both  new  starts  and  continu- 
ing programs.  This  assessment  and  appropriate  recommendations  will  be 
used  by  AFSC/XR  tc  establish  Command  positions  on  those  Advanced  and 
Engineering  Development  programs  that  should  be  pursued.  Management 
of  approved  programs  will  remain  the  responsibility  of  DL. 

3.  Your  suggestions  as  to  how  Product  Divisions  can  most  effectively 
participate  in  the  guidance  and  coordination  of  advanced  development 
programs  are  solicited.  Considerations  such  as  documentation  required, 
procedures  needed  and  Product  Division  point  of  contact  should  be 
addressed. 


4.  Request  your  comments  be  provided  by 
Primary  point  of  contact  is  Lt  Col  James 


21  February  1975  to  AFSC/{RP. 

P.  Erl,  AFSC/XRPP,  AV  858-4825; 
Sigethy,  AFSC/DLXB,  AV  858-2968. 
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lutjccT < Monitor  and  Control  of  Laboratory  Advanced/Engineering  Development 
Programs  .(AFSC/Cy.Ltr,  29.  Jan  75). 


to.  SAMSO  ASD  - ESD  ADTC  AMD  AFWAL  AFAL  AFAPL 

AFATL  AFCRl  AFFDL  AFHRl  AFML  AFRPL  AFWL  RADC 

(Commander/Di rector) 

1.  The  several  ASD  test  case  assessments  of  proposed  advanced  develop- 
ment projects-and  the  15  August  1975  "quick  response"  prioritizations: 
of  6.3  efforts  by  all  product  divisions  have  been  of  .particular  value 
These  efforts  were  not  only  helpful  in  terms,  of.  implementing  assess- 
ment policy. but iprovided  .a  valuable  contribution  to  the  programming 
and  budgeting  exercises  during  August  and  September  1975. 

2.  Based  on  our  experience  thus  far*  a more  definitive  set  of  ' 
procedures  for  the  assessment  process  has  been  formulated  (attached).  . 
Both  in-depth  assessment  of  proposed  (new)  advanced  development  projects 
and  an  annual  assessment  of  all  6.3  efforts  are  covered.  We  desire  to 
implement  the  process  immediately.  Therefore,  laboratories  will  forward 
technology  program  plans  for  proposed  FY  77  and  FY  78  new  start  advanced 
development  efforts  to  appropriate  product  division  assessment  focal 
points  (ADTC/XR,  ASD/XR,  ESD/XR,  SAMSO/XR)  as  soon  as  possible.  The 
product  divisions,  in  turn,  will  accomplish  coordinated  assessments, 

to  be  forwarded  to  AFSC/XR. 

3.  The  assessment  process  serves  a number  of  worthwhile  purposes. 

First,  it  should  encourage  early  dialogue  between  the  laboratories  and 
product  divisions.  Second,  both  initial  and  recurring  assessment 
results  will  be  of  invaluable  use  to  this  headquarters  during  the 
programming  and  budgeting  cycles.  Finally,  a continuing  involvement 
of  product  division  people  with  laboratory  programs  will  enhance  the 
successful,  transition  of  technology  to  operational  capability. 

4.  Please  provide  any  comments  on  the  new  procedures  to  AFSC/XRP. 
Contact  is  Lt  Col  0.  P.  Eri,  XRPO,  AUTOVON  858-2054. 

ROBERT  T.  MARSH,  Lt  Gen,  U5Af 
Vice  Comaander 


1 Atch 

Assessment  Procedures 
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DEPARTMENT  OF  THE  AIR  FORCE  AFSC  REGULATION  80*19 

Headquarter*  Air  Force  System*  Command 

Andrew*  Air  Force  Bate,  DC  20334  31  Match  1976 

Research  and  Development 

ASSESSMENTS  OF  ADVANCED  DEVELOPMENT  TECHNOLOGY  PROGRAMS 


This  regulation  prescribes  the  responsibilities  and  procedures  for  assessing  and  advocating  technology  base 
(nonsystems)  advanced  development  programs.  It  establishes  a uniform  policy  to  initiate,  control,  and  ad- 
vocate nonsystems  advanced  development  (6.3)  programs.  It  will  provide  an  effective  basis  for  recom- 
mending the  allocation  of  resources  and  enhance  the  transfer  of  demonstrated  technology  into  weapons 
systems  development  programs  and  into  the  operational  inventory. 


1.  Terms  Explained: 

a.  Laboratories.  Defined  as  AFATL,  AFGL, 
AFHRL,  AFRJPL,  AFWL,  AFWAL,  RADC,  and 
AMD/RD.  Additionally,  field  organizations  that 
originate  and  manage  technology  base  advanced 
development  programs  may  be  defined  as  a 
laboratory  for  this  regulation  (for  example, 
AFCEC  and  SAM  SO). 

b.  Planning  Summary.  An  expanded  program 
element,  project,  or  task  summary  of  planned  ef- 
forts. It  contains  all  data  elements  of  DD  Form 
1634,  Research  and  Development  Planning  Sum- 
mary, and  AF  Form  1337,  Weapon  System  Budget 
Estimate  Normally,  a product  of  AFSC  com- 
puterized management  information  systems  which 
provides  information  on  resources,  objective  and 
payoff,  plans,  schedule,  progress  and  accomplish- 
ments. May  be  used  instead  of  Technology 
Program  Plan  when  approval  is  indicated  on 
AFSC  Form  56,  AFSC  Program  Direction. 

c Program  Assessment  A formal  evaluation  by 
SAMSO,  ASD,  ESD,  or  ADTC  of  a proposed  or 
ongoing  program  addressing  relevance,  payofT  and 
benefit,  priority,  cost  and  schedule,  and  timeliness. 
Assessments  of  proposed  programs  will  be  an  in- 
depth  review.  SAMSO,  ASD,  ESD,  and  ADTC 
will  assess  ongoing  programs  annually  to  reap- 
praise all  efforts  previously  assessed  and  of  con- 
tinued interest  to  them. 

d Technology  Program  Plan  (TPP).  Primary 
planning  document  tailored  to  describe  and  sup- 
port the  program  concisely  and  adequately.  It  is 
written  at  the  program  element,  project,  or  task 
level  as  appropriate,  reviewed  annually,  and  up- 
dated when  required. 

2.  Policy.  AFSC  field  commands  and  laboratories 
will  originate  program  planning  documentation 
annually  in  response  to  formal  direction,  or 
establish  an  advanced  development  program  to 
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demonstrate  emerging  technology.  When  the 
documentation  is  forwarded  to  HQ.  AFSC,  copies 
will  be  sent  to  each  field  command  where  a poten- 
tial application  exists  for  an  assessment  If 
SAMSO,  ASD,  ESD,  and  ADTC  do  not  have  a 
direct  interest  a using  organization  (such  as  a 
major  command,  another  service,  or  a DOD 
agency)  will  be  requested  to  provide  an  assessment 
The  assessement  will  be  used  in  developing  an 
AFSC  position  and  for  interfacing  with  HQUSAF 
in  the  approval  and  resource  allocation  process. 

3.  Assessments  of  New  Program* 

a.  Laboratories  will  initiate  and  generate  TPPs 
for  proposed  advanced  development  programs  that 
are  either  in  response  to  program  direction  from 
HQ  AFSC  or  are  initiated  within  the  laboratory. 
The  TPP  will  concisely  describe  the  program 
payoff,  technical  approach,  management  concept 
required  resources,  schedule,  and  analysis  of 
technical/ resource/ schedule  risk-  The  specific 
weapon  system  improvements  and  development 
goals  that  the  technology  program  is  designed  to 
benefit  will  be  designated,  if  at  all  possible.  Coor- 
dination will  be  obtained  from  all  participating 
organizations.  The  TPP  will  be  forwarded  to 
AFSC/DL  and  copies  sent  to  SAMSO,  ASD,  ESD, 
or  ADTC  if  potential  applications  exist 

b.  SAMSO,  ASD,  ESD,  and  ADTC  will 
designate  a focal  point  to  assess  the  TPP  (normal))' 
the  Development  Plans  Office)  and  will  perform 
an  in-depth  assessment  of  each  factor  in  attach- 
ment 1.  While  SAMSO,  ASD,  ESD,  and  ADTC 
will  not  normally  question  the  technical  ap- 
proach, comment  based  on  experience  with  similar 
efforts  is  appropriate.  An  indorsement  of  an  effort 
implies  the  intent  to  use  successfully  demonstrated 
technology  if  die  intended  application  has  per- 
sisted SAMSO,  ASD,  ESD,  and  ADTC  will  send 
the  assessment  within  45  calendar  days  after 
receipt  to  AFSC/XR  and  a copy  to  the  TPP 
originating  organization. 

c.  HQ  AFSC  (XR)  will  send  copies  of  the 
assessment  to  AFSC/DL  and  SD.  HQ  AFSC  (XR) 
will  be  the  lead  agent  in  formulating  AFSC 
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position  on  the  proposed  programs  and  coordinate 
the  position  with  HQ,  AFSC  (DL)  and  other  DCSs 
as  appropriate.  The  AFSC  position  will  take  one  of 
two  forms: 

( 1 ) Programs  Favorably  Considered  A com* 
mand  letter  will  be  sent  to  HQ  USAF/RD 
requesting  approval  of  the  new  start  While  new 
programs  may  be  advocated  throughout  the  year, 
they  will  compete  for  available  resources  along 
with  all  other  programs  during  annual  budget  sub- 
missions. 

(2)  Programs  Not  Favorably  Considered  A 
letter,  coordinated  with  staff,  wil1  be  sent  to  the 
organization  originating  the  TPP.  The  letter  will 
explain  why  the  proposed  effort  is  not  favorably 
considered  and  will  provide  additional  instructions. 


4.  Annual  Review  and  Assessment  of  Ongoing 
Programs 

a Laboratories  will  normally  be  required  to  up- 
date TPPs  annually  to  coincide  with  requirements 
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of  the  budget  process.  In  some  cases,  only  updated 
Planning  Summaries  will  be  required  The  up- 
dated TPP  and/or  updated  Planning  Summaries 
and  Budget  Estimates  will  br  forwarded  annually 
to  AFSC/DL  by  mid-December  and  copies  sent  to 
the  assessing  divisions  (SAM SO,  ASD,  ESD,  or 
ADTC). 

b.  If  SAMSO,  ASD,  ESD,  or  ADTC  originally 
assessed  the  program,  they  will  review  the  updated 
documentation  and  confirm  or  modify  the  previous 
assessment  and  the  relative  priority  ranking.  A 
statement  of  supportability,  rationale,  and  priority 
ranking  for  each  effort  will  be  provided  annually  to 
AFSC/XR  by  31  January.  The  format  shown  in  at- 
tachment 2 will  be  used  Copies  will  be  sent  to  the 
originating  laboratories.  HQ  AFSC  (XR)  will  send 
copies  to  AFSC/DL  and  SD. 

c At  HQ  AFSC,  the  updated  TPP  and 
assessments  will  be  used  in  the  annual  budget 
determinatioa  HQ  AFSC  (DL)  will  forward  the 
TPP  to  HQ  USAF  when  required  by  PMD  in- 
structions. 


OFFICIAL 


WILLIAM  J.  EVANS,  General,  USAF 
Commander 


DAVID  M.  HUDACK,  Colonel,  USAF  2 Attachments 

Director  of  Administration  1.  General  Outline  of  Assessment  Paper 

2.  General  Outline  of  Annual  Assessment  of  Ad- 
vanced Development 
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GENERAL  OUTLINE  OF  ASSESSMENT  PAPER 
Initial  Indepth  AucMaent 

This  outline  is  designed  to  facilitate  examination  of  payoff,  relevance,  cost  implication,  timeliness,  and 
relative  priority,  Keep  the  paper  short — four  to  seven  pages.  It  must  be  clear  and  concise  without  constant 
reference  to  the  ADP  planning  documents.  Use  philosophy  of  generic  relationship  of  a main  text  to  appen- 
dices: that  is,  main  text  must  have  enough  information  to  structure  position  while  appendix  (ADP  plan- 
ning documents)  can  and  should  be  consulted  by  reader  if  detailed  background  knowledge  is  desired. 


Section  I.  Introduction  Give  a brief  history  of 
ADP.  Show  outgrowth  from  6.2  efforts;  program 
chronology,  including  any  past  attempts  to  ad- 
vocate this  approach;  reasons  for  past  disapproval 
or  deferments;  reasons  for  submittal  at  this  time. 

Section  II.  Payoff.  Payoffs  can  be  looked  at  from 
two  points  of  view:  the  effort  will  provide  an  “im- 
provement” that  coincides  with  the  schedule  of 
identifiable  applications,  or  the  eifort,  if  successful, 
will  provide  a technological  opportunity  (that  is, 
improved  capability)  of  signific?m  benefit  to  the 
Air  Force,  although  the  specific  application  cannot 
be  identified  at  this  time. 

A Potential  Applications.  Identify  and  discuss  ap- 
plication cited  in  ADP;  identify  other  applications, 
if  any.  that  present  themselves  Consider  phasing 
with  either  new  system  developments  and/or 
modifications  to  inventory  systems. 

1.  What  are  fhe  most  probable  applications? 

2.  What  is  the  likelihood  the  identified  ap- 
plications will  be  developed?  acquired? 

3.  How  important  are  the  applications  to  the  Air 
Force? 

4.  What  is  the  likelihood  the  proposed  tech- 
nology will  indeed  transition  into  identified  ap- 
plications? 

5.  Does  this  effort  directly  address  an  operational 
need  or  deficiency?  What  need?  (Refer  to  develop- 
ment goals.  ROCs.  and  so  forth.)  How  critical  is 
the  need? 

6.  Is  this  effort  a viable  solution? 

B.  Technological  Opportunities.  Identify  and 
discuss  payoff  cited  in  ADP;  identify  potential  ap- 
plications. if  possible. 

1.  What  is  the  improved  capability? 

2.  Quantify  the  performance  improvement 
projection,  if  possible. 


<j.  tt  uuiu  ii ic  r, 1 1 Force  be  likely  to 
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technology? 

4.  What  are  the  alternatives,  if  any,  to  getting  the 
same  or  better  performance  improvements? 

C.  Cost  Reduction.  Discuss  reduced  cost  aspects  of 
the  proposed  payoff,  if  applicable. 

Is  there  a potential  for  significant  cost  reduction? 


Section  III.  Schedule  Review  the  proposed 
schedule  for  realistic  achievability  and  for  com- 
patibility with  the  schedules  of  probable  ap- 
plications. 

A Is  the  program  ready  to  transition  to  an  ad- 
vanced development  effort? 

B.  Is  there  a sufficient  exploratory  development 
data  base? 

C.  Is  this  primarily  hardware-proof  versus  con- 
cept feasibility? 

D.  Are  the  program  duration  and  objectives 
clearly  defined? 

E.  Does  the  effort  duplicate  or  complement 
other  efforts?  Explaia 

(NOTE:  Technical  feasibility  should  not  be 
questioned  unless  there  are  strong  arguments 
against  the  approach;  for  example,  previous  near- 
identical experience.) 

F.  What  is  the  confidence  in  proposed  schedule 
estimates?  What  are  the  pacing  elements? 

G.  How  well  docs  the  planned  schedule  match 
the  schedule  of  potential  applications? 

K.  How  realistic  (viable)  are  application 
schedules? 

I.  Does  the  schedule  accommodate  a logical 
transition  plan  between  completion  of  advanced 
development  and  systems  application?  What  it  that 
transition  mechanism  and  its  schedule? 

(NOTE:  A very  important  part  of  the  dialogue 
that  assessments  should  promote  is  for  potential 
technology  appliers  to  carefully  review  how  far  the 
laboratory  intends  to  carry  the  6.3  effort  and  to 
provide  the  laboratory  with  recommendations  that 
would  facilitate  the  transfer  of  technology.) 

J.  Are  schedule  changes  called  for  (for  example, 
to  better  match  applications  schedules)?  What  im- 
pact will  there  be  on  funding  and  schedule  com- 

— r tr-.* -a 
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K.  Would  risk  introduced  be  acceptable? 

Section  IV.  Coil  Is  the  cost  consistent  with  the 
technical  approach;  are  the  cost  projections  ,• 
realistic;  is  the  payoff  significant  enough  to  warrant 
the  development  cost? 
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Section  V.  Priority.  Based  on  the  assessment  fac- 
tors in  the  sections  above,  indicate  relative  level  of 
importance  for  the  proposed  effort  Summarize  the 
rationale  for  the  prioritization  choice. 

What  importance  category  would  you  place  this  ef- 
fort in? 

1.  Highest  Priority.  Should  be  funded  as 
required  to  meet  schedule  recommended  in 
assessment 

2.  Middle  Priority.  The  technology  is  assessed  to 
be  important  to  the  Air  Force;  however,  schedule  is 
not  so  critical  that  funding  reductions,  with  atten- 
dant schedule  impact,  cannot  be  made. 

3.  Low  Priority.  The  assessed  value  to  the  Air 
Force  is  too  low  to  warrant  funding 


Section  VI.  Issues  Conflicts.  Identify  any  issues 
that  exist  between  the  laboratory  and  the  assessing 
organization  regarding  the  technical  approach,  the 
projected  uses  or  applications  of  the  technology, 
and  timeliness  of  the  demonstration.  Develop  the 
respective  positions. 


Section  VII.  Decision  Options.  Structure  the 
decision  options  AFSC  has  to  resolve  the  issues 
and  either  support  or  reject  the  planned  program. 


Section  VIII.  Recommendations.  Develop  the 
recommended  position. 
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AFSCR  80-19  Attachment  2 SI  March  1976 

GENERAL  OUTLINE  OF 

ANNUAL  ASSESSMENT  OF  ADVANCED  DEVELOPMENT 

PROGRAM  ELEMENT TITLE. 

PROJECT/TASK  NUMBER TITLE  

OBJECTIVE:  (Brief  explanation  of  the  project’s  goals  in  terms  of  payoff,  schedule,  and  application.) 

CATEGORY:  (Select  one  of  the  three  priorities  listed  in  attachment  1,  section  V,  using  the  stated 
criterion.) 

RATIONALE:  (Provide  appropriate  rationale  for  categorization.  It  is  important  that  a reaffirmation  of 
continued  support  as  stated  in  the  initial  indepth  assessment  be  included.  The  rationale  should  include 
factors  such  as  whether  the  schedule  is  still  “on  track”  and  in  phase  with  most-probable  application*.  The 
reason  for  changes  in  support  of  efforts  should  be  clearly  stated.) 

REFERENCE  DOCUMENTATION:  (List  the  source  documentation  used  for  the  overall  assessment,  in- 
cluding date  of  documentation) 
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APPENDIX  C 


Interview  Schedule 
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PRIVACY  STATEMENT 


A.  PRINCIPLE  PURPOSE!  THE  INTERVIEW  IS  BEING  CONDUCTED  TO  COLLECT  INFORMATION 
TO  BE  USED  IN  RESEARCH  AIMED  AT  ILLUMINATING  AND  PROVIDING  INPUTS  TO  THE 
SOLUTION  OF  PROBLEMS  OF  INTEREST  TO  THE  AIR  FORCE  AND/OR  DOD. 

B.  ROUTINE  USES!  THE  INTERVIEW  DATA  WILL  BE  ANALYZED  AND  THE  RESULTS  WILL 
BE  INCLUDED  IN  WRITTEN  MASTER'S  THESIS  AND  MAY  ALSO  BE  INCLUDED  IN  PUB- 
LISHED ARTICLES,  REPORTS,  OR  TEXTS.  DISTRIBUTION  OF  THE  RESULTS  OF  THE 
RESEARCH,  BASED  ON  THE  INTERVIEW  DATA,  WHETHER  IN  WRITTEN  FORM  OR  PRE- 
SENTED ORALLY,  WILL  BE  UNLIMITED. 

C.  PARTICIPATION  IN  THIS  EFFORT  IS  ENTIRELY  VOLUNTARY,  AND  NO  ADVERSE  ACTION 
OF  ANY  KIND  MAY  BE  TAKEN  AGAINST  ANY  INDIVIDUAL  WHO  ELECTS  NOT  TO  PARTI- 
CIPATE. 

D.  YOUR  RESPONSES  AND  COMMENTS  WILL  BE  CONSIDERED  ANONYMOUS. 


INTERVIEW  $ DATE 


CSM/SM/76S-21 


INTERVIEW  SCHEDULE 


BACKGROUND  INFORMATION 


1.  ARE  YOU  PRESENTLY  ASSIGNED  TO  A 


LABORATORY 
PRODUCT  DIVISION 
PRODUCT  DIVISION 
PRODUCT  DIVISION 
OTHER 


(STAFF  ORGANIZATION) 
(PROGRAM  OFFICE) 
(FUNCTIONAL  ORGANIZATION) 


2.  WHAT  IS  YOUR  GRADE?  

3.  HOW  LONG  HAVE  YOU  WORKED  IN  THE  RESEARCH  $,  DEVELOPMENT  FIELD? 

LESS  THAN  1 YR  12  YRS  TO  16  YRS 

1 YR  TO  4 YRS  16  YRS  TO  20  YRS 

~ 4 YRS  TO  8 YRS  MORE  THAN  20  YRS 

8 YRS  TO  12  YRS 

4.  HOW  LONG  HAVE  YOU  BEEN  IN  YOUR  PRESENT  JOB  ASSIGNMENT? 


LESS  THAN  6 MOS 
6 MOS  TO  2 YRS 
2 YRS  TO  4 YRS 
4 YRS  TO  6 YRS 


6 YRS  TO  8 YRS 
8 YRS  TO  12  YRS 
MORE  THAN  12  YRS 


5.  IF  YOU  WORK  IN  A PRODUCT  DIVISION,  PLEASE  SELECT  THE  ONE  STATE- 
MENT WHICH  COMES  CLOSEST  TO  REPRESENTING  THE  DEGREE  TO  WHICH 
YOU  HAVE  PARTICIPATED  IN  ADP  ASSESSMENTS.  I HAVE: 


BEEN  A FOCAL  POINT  FOR  GATHERING  COMMENTS 
FOR  ADP  ASSESSMENTS. 

PARTICIPATED  JN  THE  ACTUAL  WRITING  OF  AN 
ADP  ASSESSMENT. 

PROVIDED  WRITTEN  COMMENTS  TO  SUPPORT  THE 
ASSESSMENT  OF  MORE  THAN  ONE  ADP. 

PROVIDED  WRITTEN  COMMENTS  TO  SUPPORT  THE 
ASSESSMENT  OF  JUST  ONF  ADP. 

NOT  PERSONALLY  PROVIDED  WRITTEN  COMMENTS, 

BUT  HAVE  PARTICIPATED  IN  THE  OFFICE  ASSESSMENT 
OF  AN  ADP. 

NOT  PARTICIPATED  IN  AN  ASSESSMENT,  BUT  AM 
KNOWLEDGEABLE  OF  THE'  AD?  ASSESSMENT  REQUIRE- 
MENT. 

NOT  PARTICIPATED  IN  AN  ASSESSMENT,  AW  AM 
ONLY  VAGUELY  AWARE  OF  THE  ADP  ASSESSMENT 
REQUIREMENT. 

NOT  HEARD  OF  THE  ADP  ASSESSMENT  REQUIREMENT 
BEFORE  TODAY. 
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THE  REQUIREMENT  FUR  PRODUCT  DIVISIONS  TO  ASSESS  ADVANCED  DEVELOPMENT 
PROGRAMS  (ADP)  CONDUCTED  BY  THE  LABORATORIES  WAS  ESTABLISHED  BY  HQ  AFSC  IN 
DEC  1975.  ONE  GOAL  OF  THIS  POLICY  WAS  TO  INCREASE  THE  TRANSFER  OF  ADT  TECH- 
NOLOGY TO  SYSTEM  APPLICATION. 

IN  MAR  1974,  A STUDY  WHICH  ADDRESSED  THE  TRANSFER  OF  TECHNOLOGY  FROM 
FEDERAL  LABORATORIES  WAS  CONDUCTED  BY  THE  SYRACUSE  UNIVERSITY  RESEARCH  CORP. 

FOR  THE  NATIONAL  SCIENCE  FOUNDATION.  ONE  CONCLUSION  OF  THE  STUDY  GROUP  WAS 
THAT  THE  OCCURRENCE  OF  THE  FOLLOWING  EVENTS  GREATLY  ENHANCES  THE  TRANSFER 
ABILITY  OF  A TECHNOLOGY. 

(1)  CONDUCT  OF  PLANNING  ACTIVITIES  WHICH  SERVE  TO  EXPOSE  POTENTIAL 
PROBLEMS  BEFORE  THEY'  BECOME  SERIOUS. 

(2)  THE  ESTABLISHMENT  OF  A FORUM  WITHIN  WHICH  PROBLEMS  MAY  BE  RESOLVED 
AS  THEY  COME  ALONG. 

(3)  THE  EMERGENCE  OF  STRONG  MANUFACTURER  (e.g.,  PRODUCT  DIVISION) 

SUPPORT  OF  A TECHNOLOGY  RESULTING  IN  A BALANCE  BETWEEN  "TECHNOLOGY 
PUSH"  AND  "DEMAND  PULL." 

(4)  ESTABLISH  THAT  THE  TECHNOLOGY  IS  SUFFICIENTLY  MATURE  SO  THAT 
THOSE  INVOLVED  CAN  ‘L?  THAT  THE  REMAINING  TECHNICAL  AND  COST  BARRIERS 
ARE  SOLUBLE. 

(5)  THE  EXISTANCE  OF  AN  Tfi  ' xNFOSHP  rONAL  LINKAGE  BETWEEN  THE 
INNOVATOR  (e.g.,  THE  *ABOK-  1Y)  AND  _HE  NEEDER  (e.g.,  THE  PRODUCT 
DIVISION  AND/OR  THE  USER). 

6.  PLEASE  INDICATE  THE  EXTENT  THAT  YOU  AGREE  OR  DISAGREE  WITH  THE  STATEMENT 

THAT  THE  ADP  ASSESSMENT  POLICY  WILL  ENCOURAGE  THE  OCCURRENCE  OF  SUCH  EVENTS. 

STRONGLY  DISAGREE  INCLINED  UNDECIDFU  INCLINED  AGREE  STRONGLY 
DISAGREE  TO  DISAGREE  TO  AGREE  AGREE 

(1)  PLANNING  ACTIV- 
ITIES WHICH  MAY 

EXPOSE  PROBLEMS 

(2)  PROVIDE  A 
FORUM  TOR 
PROBLEM 

RESOLUTION  

(3)  STRONG  PRODUCT 
DIVISION 

SUPPORT  

(4)  ESTABLISH  THAT 
TECHNOLOGY  IS 

MATURE  

(5)  ESTABLISH 
AN  EA.RLY 
INFORMATIONAL 
LINKAGE 


USE  REVERSE  FOR  COMMENTS. 
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OTHER  STUDIES  SPONSORED  BY  THE  NATIONAL  SCIENCE  FOUNDATION  HAVE  IDEN- 
TIFIED THE  FOLLOWING  FACTORS  AS  TYPICAL  BARRIERS  TO  TECHNOLOGY  TRANSFER 
BETWEEN  AGENCIES*  (l)  INERTIA  BARRIERS,  (2)  LACK  OF  AN  INCENTIVE  STRUCTURE, 

(3)  COST  BARRIERS,  (A)  COMMUNICATION  BARRIERS,  (5)  TIME  BARRIERS,  (6)  GEO- 
GRAPHIC DISTANCE,  (7)  NONEXISTENT  TRANSFER  MANAGEMENT  STRUCTURE,  (8)  TECH- 
NOLOGY BARRIERS. 

IN  THE  NEXT  TWO  QUESTIONS  PLEASE  INDICATE  THE  EXTENT  THAT  YOU  AGREE  OR 
DISAGREE  WITH  THE  STATEMENT  THAT  A GIVEN  FACTOR  REPRESENTS  A BARRIER  TO  THE 
TRANSFER  OF  ADP  TECHNOLOGY  TO  SYSTEM  APPLICATION. 

7.  BEFORE  THE  ADP  ASSESSMENT  POLICY  WENT  INTO  EFFECT,  THE  FOLLOWING  FACTORS 
REPRESENTED  A BARRIER  TO  THE  TRANSFER  OF  ADP  TECHNOLOGY  TO  SYSTEM  APPLI- 
CATION. 

STRONGLY  DISAGREE  INCLINED  UNDECIDED  INCLINED  AGREE  STRONGLY 
DISAGREE  TO  DISAGREE  TO  AGREE  AGREE  l 

(1)  INERTIA 
BARRIERS 


(2)  LACK  OF 
INCENTIVE 
STRUCTURE 


(3)  COST 

BARRIERS 


(4)  COMMUNICATION 
BARRIERS 


(5)  TIME 
BARRIERS 


(6)  GEOGRAPHIC 
DISTANCE 


(7)  TRANSFER 
MANAGEMENT 
STRUCTURE 


(8)  TECHNOLOGY 
BARRIERS 


COMMENTS: 


/ 
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8.  NOW  THAT  THE  ADP  ASSESSMENT  POLICY  IS  IN  EFFECT,  THE  FOLLOWING  FACTORS 
WILL  REPRESENT  (OR  "STILL  REPRESENT")  A BARRIER  TO  THE  TRANSFER  OF  ADP 
TECHNOLOGY  TO  SYSTEM  APPLICATION. 

STRONGLY  DISAGREE  INCLINED  UNDECIDED  INCLINED  AGREE  STRONGLY 
DISAGREE  TO  DISAGREE  TO  AGREE  AGREE 

(1)  INERTIA 
BARRIERS 


(2)  LACK  OF 
INCENTIVE 
STRUCTURE 


(3)  COST 
BARRIERS 


(A)  COMMUNICATION 
BARRIERS 


(5)  TIME 
BARRIERS 


(6)  GEOGRAPHIC 
DISTANCE 


(7)  TRANSFER 

MANAGEMENT  V . 

STRUCTURE 


(8)  TECHNOLOGY 
BARRIERS 


COMMENTS: 
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ONE  GOAL  OF  THE  ADP  ASSESSMENT  POLICY  IS  TO  INCREASE  DIALOGUE  AND  COM- 
MUNICATION BETWEEN  THE  PRODUCT  DIVISIONS  AND  LABORATORIES.  THE  NEXT  SET 
OF  QUESTIONS  IS  DESIGNED  TO  MEASURE  HOW  WELL  THIS  GOAL  IS  BEING  ACHIEVED. 

9.  BELOW  IS  A LIST  OF  ADVANCED  DEVELOPMENT  PROGRAMS/ PROJECTS  EITHER  BEING 
CONSIDERED  OR  CURRENTLY  ONGOING  IN  THE  LABORATORIES.  PLEASE  PLACE  A 
CHECK  MARK  BESIDE  A PROGRAM/ PROJECT  IF  YOU  LEARNED  OF  IT  BECAUSE  OF 
THE  ADP  ASSESSMENT  POLICY/ACTIVITY. 


NOTE:  THE  ORIGNIAL  INTERVIEW  SCHEDULE  CONTAINED 

A LIST  OF  FIFTEEN  ADVANCED  DEVELOPMENT 
TECHNOLOGY  PROGRAMS.  THIS  LIST  HAS  BEEN 
DELETED  IN  ORDER  TO  PRESERVE  THE  ANONYMITY 
OF  THE  LABORATORIES  AND  PRODUCT  DIVISIONS 
VISITED. 


10.  CAN  YOU  THINK  OF  ANY  OTHER  ADVANCED  DEVELOPMENT  PROGRAMS/PROJECTS  THAT 
YOU  NOW  KNOW  ABOUT,  BUT  PROBABLY  WOULD  NOT  HAVE  LEARNED  OF  IF  THE  ADP 
ASSESSMENT  POLICY/ACTIVITY  HAD  NOT  OCCURRED? 

YES  NO 


IF  MORE  THAN  ONE,  ABOUT  HOW  MANY? 
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11.  WHAT  IS  YOUR  BEST  ESTIMATE  OF  THE  PERCENT  OF  YOUR  WORKING  TIME  THAT  WAS 
SPENT  COMMUNICATING  WITH  PERSONNEL  IN  THE  AGENCIES  LISTED  BELOW  REGARDING 
NEW  ADPs  BEFORE  THE  ADP  ASSESSMENT  REQUIREMENT  WENT  INTO  EFFECT? 

(1)  HIGHER  HEADQUARTERS 

(2)  PRODUCT  DIVISIONS 

(3)  LABORATORIES 

(4)  AFLC 

(5)  OPERATIONAL  COMMANDS 

12.  WHAT  IS  YOUR  BEST  ESTIMATE  OF  THE  PERCENT  OF  YOUR  WORKING  TIME  THAT  IS 
OR  WILL  BE  SPENT  COMMUNICATING  WITH  PERSONNEL  IN  THE  AGENCIES  LISTED 
BELOW  REGARDING  NEW  ADPs  NOW  THAT  THE  ADP  ASSESSMENT  REQUIREMENT  IS  IN 
EFFECT? 

Cl)  HIGHER  HEADQUARTERS 

(2)  PRODUCT  DIVISIONS 

(3)  LABORATORIES 

(4)  AFLC 

(5)  OPERATIONAL  COMMANDS  1- 

13.  HOW  MUCH  OF  YOUR  WORKING  TIME  SHOULD  BE  SPENT  COMMUNICATING  WITH  PERSONNEL 
IN  THE  LABORATORY  (PRODUCT  DIVISION)  IN  ORDER  FOR  YOU  TO  STAY  AWARE  OF 
STATUS  OF  ADVANCED  DEVELOPMENT  PROGRAMS  (SYSTEMS)  WHICH  ARE  RELEVANT 

TO  YOUR  JOB? 

X 

14.  DO  YOU  THINK  THAT  THE  ADP  ASSESSMENT  POLICY  WILL  IMPROVE  THE  DIALOGUE 
BETWEEN  YOURSELF  AND  THE  PRODUCT  DIVISION  (LABORATORY)? 

(1)  YES 

(2)  NO 

COMM El 
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TRE  HQ  AFSC  GUIDELINES  FOR  PERFORMING  THE  ASSESSMENT  OF  A NEW  ADP  REQUIRES 
SEVERAL  ITEMS  TO  BE  CONSIDERED i PAYOFF  OF  THE  ADP,  THE  FEASIBILITY  OF  THE 
SCHEDULE  OF  THE  NEW  ADP,  THE  COMPATIBILITY  OF  THE  SCHEDULE  OF  THE  NEW  ADP 
WITH  SCHEDULES  OF  PROBABLE  APPLICATIONS,  REALISTIC  COST  ESTIMATES,  PRIORITY, 
ISSUES/CONFLICTS,  DECISION  OPTIONS,  & RECOMMENDATIONS. 

15a,  IF  YOU  WORK  IN  A LABORATORY,  HOW  DIFFICULT  IS  IT  (OR  "WOULD  IT  BE")  FOR 
YOU  TO  OBTAIN  WHAT  YOU  FEEL  IS  RELIABLE  INFORMATION  ON  EACH  OF  THE  FOL- 
LOWING ITEMS  TO  INCLUDE  IN  THE  TECHNOLOGY  PROGRAM  PLAN? 

PAYOFF  SCHEDULES  ADP  COST  PRIORITY  ALTERNATE 

ESTIMATES  APPROACHES 

VIRTUALLY 

IMPOSSIBLE  


VERY  DIFFICULT 


SOMEWHAT 

DIFFICULT 


NEITHER 
DIFFICULT 
NOR  EASY 


SOMEWHAT  EASY 


EASY 


VERY  EASY 


COMMENTS: 


15b.  DO  YOU  THINK  THAT  THE  PRODUCT  DIVISIONS  HAVE  ENOUGH  TIME  AND  MANPOWER 
TO  DEVOTE  TO  THE  ASSESSMENT  PROCESS? 

YES  NO 


COMMENTS: 
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16a.  IF  YOU  WORK  IN  A PRODUCT  DIVISION,  HOW  DIFFICULT  IS  IT  (OX  "WOULD  IT  BE”) 

FOR  YOU  TO  PERFORM  WHAT  YOU  FEEL  IS  A RELIABLE  ASSESSMENT  OF  AN  ADP  WITH 
RESPECT  TO  EACH  OF  THE  FOLLOWING  ITEMS? 

PAYOFF  SCHEDULES  ADP  COST  PRIORITY  DECISION 

ESTIMATES  OPTIONS 


VIRTUALLY 

IMPOSSIBLE 

VERY  DIFFICULT 

SOMEWHAT 

DIFFICULT 

NEITHER 
DIFFICULT 
NOR  EASY 

SOMEWHAT  EASY 

EASY 

VERY  EASY 
COMMENTS t 


16b.  DO  YOU  THINK  THAT  THE  PRODUCT  DIVISIONS  HAVE  ENOUGH  TIME  AND  MANPOWER 
TO  DEVOTE  TO  THE  ASSESSMENT  PROCESS? 

YES  NO 


COMMENTS t 
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17.  ANOTHER  GOAL  OF  THE  ADP  ASSESSMENT  POLICY  IS  TO  INCREASE  THE  RESPON- 
SIVENESS OF  THE  LABORATORIES  TO  THE  PRODUCT  DIVISIONS  WITH  RESPECT  TO 
THE  CONDUCT  OF  ADVANCED  DEVELOPMENT  PROGRAMS.  PLEASE  INDICATE  YOUR 
THOUGHT  ON  THE  DEGREE  OF  RESPONSIVENESS  THAT  SHOULD  EXIST  AND  WHAT  WILL 
EXIST  NOW  THAT  THE  ADP  ASSESSMENT  POLICY  IS  IN  EFFECT. 

WHAT  SHOULD  BE  WHAT  WILL  BE 

(1)  NONRESPONSIVE 

(2)  NEITHER  RESPONSIVE  NOR  NON- 
RESPONSIVE 

(3)  TO  A SMALL  EXTENT  RESPONSIVE 

(4)  SOMEWHAT  RESPONSIVE 

(5)  FOR  THE  MOST  PART  RESPONSIVE 

(6)  VERY  RESPONSIVE 

(7)  EXCEEDINGLY  RESPONSIVE 

18.  THE  ADP  ASSESSMENT  POLICY  IS  A GOOD  IDEA. 

(1)  STRONGLY  DISAGREE 

(2)  DISAGREE 

(3)  INCLINED  TO  DISAGREE 

(4)  UNDECIDED 

(5)  INCLINED  TO  AGREE 

(6)  AGREE 

(7)  STRONGLY  AGREE 


COMMENTS i 
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INCREASING  INSTITUTIONAL  MOTIVATION  AND  THE  ESTABLISHMENT  OF  A TECHNOLOGY 
TRANSFER  AGENCY  HAVE  BEEN  IDENTIFIED  BY  NON-DOD  STUDIES  AS  NECESSARY  ACTIONS 
REQUIRED  TO  IMPROVE  THE  TRANSFER  OF  TECHNOLOGY  FROM  AN  INNOVATING  ORGANIZATION 
TO  THE  USER.  BELOW  ARE  SOME  IDEAS  WHICH  MIGHT  BE  USED  WITHIN  THE  AIR  FORCE 
TO  ACCOMPLISH  THESE  TWO  OBJECTIVES. 

(1)  HAVE  THE  PRODUCT  DIVISIONS  ANNUALLY  RATE  THE  LABORATORIES  WITH 
RESPECT  TO  GENERATING  TRANSFERABLE  TECHNOLOGY. 

(2)  HAVE  THE  LABORATORIES  ANNUALLY  RATE  THE  PRODUCT  DIVISIONS  WITH 
RESPECT  TO  USING  THE  TECHNOLOGY  THAT  THE  LABORATORIES  GENERATE. 

(3)  BOTH  (1)  AND  (2). 

(A)  SOMEHOW  MAKE  FUNDING  FOR  FUTURE  ADPs  CONDUCTED  BY  THE  LABORATORIES 
CONTINGENT  UPON  THE  COMPLETION  OF  TECHNOLOGY  TRANSFER  IN  THE  PAST. 

(5)  ESTABLISH  A TECHNOLOGY  TRANSFER  AGENCY  STAFFED  BY  PROFESSIONALS 
IN  TECHNOLOGY  TRANSFER.  THIS  AGENCY  WOULD  DEVELOP  A STRATEGIC  PLAN, 

STRUCTURE  ORGANIZATIONAL  RELATIONS,  ACT  AS  A PRODUCT  ADVOCATE,  AND 
INTERVENE  AT  KEY  POINTS  (WITH  FUNDS  IF  NECESSARY).  THE  AGENCY  WOULD 
ALSO  HAVE  TO  BE  WILLING  TO  RISK  FAILURE  AND  NOT  BE  UNDULY  CRITICIZED 
WHEN  A NEW  TECHNOLOGY  PROGRAM  DOES  NOT  PROVE  SUCCESSFUL. 

19.  PLEASE  INDICATE  THE  EXTENT  THAT  YOU  AGREE  OR  DISAGREE  THAT  THESE  IDEAS 
HAVE  ENOUGH  MERIT  TO  WARRENT  FURTHER  CONSIDERATION. 

STRONGLY  DISAGREE  INCLINED  UNDECIDED  INCLINED  AGREE  STRONGLY 
DISAGREE  TO  DISAGREE  TO  AGREE  AGREE 

(1)  PRODUCT 
DIVISIONS 

RATE  LABS  

(2)  LABS  RATE 
PRODUCT 

DIVISIONS  

(3)  BOTH  (1) 

AND  (2)  

(A)  MAKE  FUTURE 
FUNDING  CON- 
TINGENT PAST 

TRANSFERS  

(5)  ESTABLISH 
TECHNOLOGY 
TRANSFER 
AGENCY 


USE  REVERSE  FOR  COMMENTS. 
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DEFINITION  OF 
PROPOSED  BARRIERS 


(1)  INERTIA  BARRIERS 

Product  too  different  from  what  has  been  done  in  the  past. 

A predominant  commitment  to  "current  technology"  rather  than  sup- 
porting advanced  technology  concepts  and  subsystem. 

Uncertainty  of  internal  predictions  of  cost,  schedule,  and  performance. 

(2)  LACK  OF  AN  INCENTIVE  STRUCTURE 

Advanced  Development  Program  funding  technique. 

Discouragement  of  long  term  efforts. 

Individual  fear  of  being  blamed  for  failure. 

Lack  of  a laboratory  performance  evaluation  system. 

(3)  COST  BARRIERS 

Unavailability  of  cost  information  critical  to  decision  making. 
Predicted  ADP  cost  too  high. 

Nc  way  to  evaluate  expected  "return-on-investment"  of  an  ADP. 

Excessive  cost  required  to  "re-engineer"  an  ADP  product  before 
it  can  be  used  in  a system  application. 

Lack  of  funds  to  support  an  ADP,  including  contract  funding. 

(4)  COMMUNICATION  BARRIERS 

Lack  of  coordination  among  various  agencies  or  offices  (e.g.,  staff 
with  engineering,  laboratories  with  product  divisions,  etc.). 

Functional  specialist  have  inadequate  understanding  of  other 
function  (e.g.,  engineering  or  laboratory  or  vice  versa). 

Customer  needs  cannot  be  easily  translated  into  an  ADP  definition. 
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(5)  TIM!':  BARRIERS 


Unavailability  of  schedule  information  critical  to  decision  making. 


Lack  of  sufficient  calender  time  to  complete  ADP. 

Conflict  between  time  horizon  of  product  division  customers  and 
the  laboratories. 

High  risk  of  early  obsolescence. 

(6)  GEOGRAPHIC  DISTANCE 

Distance  between  product  division  and  laboratory. 

Distance  between  ADP  contractor  and  laboratory. 

(7)  NONEXISTENT  TRANSFER  MANAGEMENT  STRUCTURE 


•I  I 

M \ 

■:vm  ! 


Lack  of  clear  procedures  and  policies  for  approving  and/or  re- 
viewing an  ADP. 

Lack  of  a specific  policy  to  insure  transition  of  laboratory  devel- 
oped equipment  to  the  product  divisions. 

Existence  of  a prevalent  "NIH"  syndrome. 

No  mechanism  for  the  transfer  of  technical  knowledge. 

Limited  mobility  of  individuals. 

(8)  TECHNOLOGY  BARRIERS 

Technology  base  not  suitable  for  advanced  development. 

Unavailability  of  information  critical  to  predicting  the  performance 
of  an  ADP  product. 
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APPENDIX  D 


Interview  Data 
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APPENDIX  E 

Subjective  Comments  Collected 
During  Interviews 
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Introduction 


The  subjective  comments  received  during  the  conduct  of  the  inter- 
views are  presented  in  this  appendix.  The  comments  have  been  segregated 
according  to  the  individual's  organizational  grouping — laboratory,  product 
division,  and  other.  Also,  the  comments  are  listed  according  to  the  inter- 
view question  which  provided  the  stimulus  for  the  comment.  Both  verbal 
and  written  comments  were  received  during  the  conduct  of  the  interviews. 
Also,  some  individuals  offered  several  comments  regarding  one  question. 

An  attempt  has  been  made  to  indicate  this  to  the  reader  by  placing 
the  comments  presented  in  this  appendix  in  a standard  conversation  format. 

If  a verbal  comment  was  made,  then  the  wording  shown  has  been  paraphrased 
by  the  researcher,  and  the  comment  is  not  enclosed  in  quotation  marks. 

If  the  comment  was  written  on  the  interview  schedule  by  the  person  inter- 
viewed, then  it  is  transcribed  exactly  as  written  and  is  enclosed  in  quo- 
tation marks.  Each  comment  received  has  been  given  a paragraph  indentation. 
However,  if  these  comments  ware  received  from  the  same  individual  then  a 
single  typewriter  space  separates  the  comments.  Conversely,  if  the  comments 
were  received  from  different  personnel,  then  a double  typewriter  space 
separates  the  comments. 
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QUESTION  6 

LABORATORY  PERSONNEL  COMMENTS 


In  order  to  determine  that  a technology  is  mature,  you  have 
to  spend  time  looking  at  it.  I don't  think  that  the  product  divisions 
have  enough  time  to  do  this. 

Management  people  don't  pay  attention  to  the  TNs  and  coupling 
meetings.  It  is  left  up  to  the  working  level,  and  the  discussions 
quickly  focus  on  more  mundane  items,  like  what  equation  will  you 
use  or  something  like  that. 

One  of  the  major  results  of  this  procedure  will  be  a formal 
acknowledgement  of  product  division  need.  Corporate  memory  will  be 
documented. 

The  establishment  that  a technology  is  mature  sometimes  be- 
comes a subjective  decision.  The  labs  say  a technology  is  mature, 
and  the  receivers  say  it  is  not. 

One  thing  you  should  be  aware  of  is  that  there  has  been  an 
unofficial  assessment  in  effect.  When  we  are  comtemplating  a new 
ADP,  we  generally  try  to  go  out  and  talk  to  people — get  their  ideas 
and  comments. 

How  well  these  things  are  accomplished  depends  on  how  severely 
they  get  involved.  They  don't  have  the  time  and  the  people.  I'm  not 
negative  toward  the  assessment  concept,  but  it  could  be  improved.  I 
hope  they  are  not  going  to  use  the  assessment  results  as  gospel  the 
first  time  around.  We  weren't  really  ready  to  support  the  assess- 
ment process.  We  had  a very  limited  amount  of  information  available 
for  the  product  divisions  to  assess. 

Right  now  we  tend  to  work  all  the  problems  too  early.  "The 
present  policy  is  strong  with  respect  to  early  identification  of  the 
mental  set  or  customer  bias  which  must  be  overcome  before  transition 
can  oc.cur--it  is  of  negative  value  in  identification  and  working  of 
the  central  technology  questions  (scientific  and  technical)  associ- 
ated with  the  new  area  in  that  the  customer  tends  to  overemphasize 
mass  production/ owner ship  considerations." 
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PRODUCT  DIVISION  PERSONNEL  COMMENTS 


You  have  to  recognize  that  there  is  not  a single  spokesman 
for  the  product  division.  The  system  acquisition  people  want  zero 
risk  and  low  cost.  The  system  engineering  people  are  concerned  with 
today’s  problems#  and  they  don’t  necessarily  want  new  technology  to 
solve  them.  The  development  planning  people  are  more  visionary  and 
are  not  as  tied  to  today's  problems.  So  you  stand  a good  chance  of 
getting  more  than  one  assessment  of  an  AD?. 

I don't  think  you  can  stress  the  need  for  an  early  informa- 
tional linkage  enough.  Changes  in  technology  tend  to  jump  in  spurts, 
thus  the  assessment  will  help  keep  the  engineering  people  abreast  of 
these  changes.  Product  division  support  depends  on  how  the  ADP  emerges — 
whether  it  addresses  a broad  or  specific  need. 

The  labs  are  not  necessarily  the  innovators.  A lot  of  ideas 
are  generated  in  the  product  divisions  through  TNs  and  through  in- 
formal discussions  with  lab  people.  The  development  planning  people 
in  the  product  divisions  sometimes  push  the  labs  into  technology  pro- 
jects. 

Establishing  that  technology  is  mature  is  the  guts  of  the 
assessment.  It  gives  you  the  indication  of  the  output,  when  and  how 
and  what  they  can  do — if  it  is  done  right  that  is.  It  will  make 
planners  write  plans.  Writing  requires  a lot  more  effort  than  just 
thinking  about  it. 

The  weakest  part  is  that  you  are  considering  each  ADP  by 
itself.  You  need  to  look  at  the  situation  on  an  interlab  basis. 

Otherwise  you  get  the  three  blind  men  and  the  elephant  effect. 

The  general  attitude  in  this  product  division  is  that  the 
labs  are  about  5 years  behind  us. 

The  problems  that  have  to  be  resolved  are  not  necessarily 
between  the  product  divisions  and  the  labs.  One  of  our  biggest  pro- 
blems is  getting  disagreements  resolved  in-house. 

Chronologically  ADPs  should  come  after  TNs.  We  can  use  the 
ADTP  assessment  to  see  if  the  labs  are  being  responsive  to  TNs. 
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QUESTION  7 

LABORATORY  PERSONNEL  COMMENTS 


You  should  add  another  barrier — too  much  competition  and 
proliferation  of  responsibility/ development/ acquisition  of  elec- 
tronic subsystems. 

1 disagree  that  we  have  a lack  of  incentive  to  wean  the 
technology.  The  lack  of  incentive  is  for  the  product  division  to 
try  to  use  it. 

Technology  barriers  are  not  a problem.  Something  good  will 
always  come  out  of  an  ADP,  or  else  you  won’t  be  able  to  sell  it. 
You'll  never  get  it  through  the  coordination  cycle. 

A transfer  management  structure  is  hard  to  define.  We’ve 
been  working  almost  two  years  on  a MOA,  and  we  finally  got  down  to 
the  nitpicking  level  and  never  did  get  it  finished. 
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QUESTION  7 

PRODUCT  DIVISION  PERSONNEL  COMMENTS 
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The  big  problem  in  answering  these  questions  is  that  there 
hasn't  been  enough  time  and  enough  assessments  accomplished  to  deter- 
mine what  the  effect  will  be. 


I don't  think  it  is  inertia  as  much  as  it  is  a conflict  in 
requirements.  For  example,  the  labs  keep  trying  to  build  an  inertial 
guidance  set  with  better  accuracy  while  the  real  system  need  is  one 
with  high  reliability,  low  cost,  and  medium  accuracy.  The  labs  need 
to  get  more  involved  with  the  everyday  mundane  things  rather  than  t’he 
high  technology  interesting  ones. 

The  technology  barriers  have  never  been  a problem. 

In  order  to  get  an  ADP  approved,  you  have  to  go  and  "regc" 
through  the  approval  cycle,  and  you  need  a favorable  political  sit- 
uation. 

The  management  structure  has  nothing  to  do  with  it.  The 
product  is  either  so  good  that  it  sells  itself,  or  somebody  within 
the  product  division  has  to  accept  the  risk  and  specify  it  in. 

The  SPO  director  is  the  big  factor  [with  respect  to  using 
an  ADTP  generated  technology].  It  depends  on  how  he  sees  the  -isk 
involved,  the  cost,  and  schedule  impact. 

Tie  labs  come  out  with  reports  that  nobody  can  use.  The 
problem  is  that  the  labs  can't  get  money  unless  they  identify  what 
weapon  system  they  are  trying  to  apply  their  program  to.  They  have 
to  sell  it  by  attaching  it  to  the  current  buzz  word.  As  a result 
they  don't  get  started  in  areas  that  are  the  primary  need.  By  the 
time  the  lab  is  aware  of  what  the  problem  is,  the  SPO  and  the  prime 
contractor  are  off  doing  the  work. 


not. 


There  is  usually  no  grave  doubt  whether  an  ADP  will  work  or 


I think  that  the  people  that  I work  with  are  very  communi- 
cative. This  includes  everyone  from  Lieutenants  to  Generals.  They 
make  it  a point  to  advertise. 

I think  people  are  more  likely  to  be  the  element  of  transfer 
than  documents. 


You  are  making  the  assumption  that  technology  transfer  is 
inherently  good.  It  could  be  good  not  to  transfer  a bad  technology. 

The  time  barrier  is  important.  The  technology  window  is 
pretty  narrow  sometimes.  He  think  the  labs  are  consistently  lobbing 
their  shells  behind  it. 

The  real  flow  [of  technology]  is  through  a contractor.  Con- 
tractors judge  whether  or  not  a new  technology  should  be  proposed. 

I don't  know  how  you  could  have  a structure  dictate  to  a contractor 
what  technology  to  use. 
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Major  transfers  do  not  occur  between  the  lab  and  the  product 
division  directly.  The  product  division  will  incorporate  a new  tech- 
nology bas^d  on  what  it  learns  during  source  selection  from  contrac- 
tors. The  labs  do  not  necessarily  have  all  the  solutions}  in  fact, 
the  contractors  will  stress  a solution  which  may  have  a little  less 
performance  and  risk  but  is  a lot  cheaper. 
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QUESTION  7 

OTHER  PERSONNEL  COMMENTS 


"Too  often  technologist  is  pursuing  technology  for  tech- 
nology's sake  and  does  not  'advertise*  the  good  work  he  is  doing." 

"Systems  too  often  take  across-the-board  cuts  in  all  tech- 
nology programs  without  sufficient  regard  for  impact  on  individuals 
task's  opportunity  windows." 
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QUESTION  8 

PRODUCT  DIVISION  PERSONNEL  COMMENTS 


To  get  good  cooperation  between  the  product  division  and 
the  labs — it  is  highly  dependent  on  the  quality  of  people  involved 
and  not  on  the  management  structure.  You  can  go  through  the  numbers 
and  add  all  the  management  structure  you  want  to,  but  in  the  end  it 
is  the  people  who  make  the  thing  work. 

It  will  let  us  police  the  idea  of  design  criteria  and  assess- 
ments. It  will  establish  what  it  will  take  to  transition — what  it 
will  take  to  sell  it — the  level  of  hardware  development,  etc. 

[The  respondent  marked  the  "disagree’'  for  all  barriers  and 
then  added  the  following  comments]  "Evidence  to  date  suggests  that 
the  above  viewpoints  are  very  optimistic."  I sent  a rough  draft  of 
my  assessment  to  the  laboratory  and  asked  for  their  comments,  and 
they  didn’t  respond. 

More  time  (about  two  years)  is  required  to  determine  if  the 
new  assessment  procedure  is  a success.  Ultimately  the  assessments 
will  determine  if  the  labs  are  doing  a more  creditable  job  of  estab- 
lishing a data  base.  This  has  been  lacking  in  the  past. 

You  need  to  do  more  than  just  ouput  hardware.  The  labs 
should  come  up  with  data  for  design  handbook  and  spec  changes.  We 
have  to  answer  the  question  what  do  we  do  to  apply  it  to  a system. 
Generally,  a specific  system  will  have  a specific  constraint  rather 
than  a generic  constraint  You  might  use  the  same  technology  in 
weapon  system  A and  weapon  system  B,  but  more  likely  the  black  boxes 
will  be  different. 


237 


mtr 


• /a 


■am  i.n7Ay.l'  rrrrr 


- .-4  • 


- -"raw 


ft  J 


GSM/SM/76S-21 


QUESTION  8 


i 


OTHER  PERSONNEL  COMMENTS 


There  is  a lot  more  to  transf erring  technology  than  over- 
coming barriers  between  the  laboratories  and  the  product  divisions. 
There  are  many  external  factors  driving  a program  manager.  He  is 
worried  about  costs,  schedule,  and  performance,  and  sometimes  he  has 
to  bite  the  bullet.  He  also  has  to  be  careful  because  if  he  says 
that  the  ADP  should  be  used  in  his  system  the  labs  will  try  to  bill 
him  for  it.  Or  someone  will  think  that  the  program  is  in  trouble 
and  try  to  cancel  it. 
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QUESTION  9 

PRODUCT  DIVISION  PERSONNEL  COMMENTS 


u 


This  list  of  programs  is  a little  unfair.  Many  of  the  pro- 
grams were  on-going  before  the  assessment  policy  went  into  effect. 


I 


ij 
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QUESTION  14 

LABORATORY  PERSONNEL  COMMENTS 


"Yes,  if  knowledgeable  working  level  engineers’  assessments 
of  ADP  programs  are  utilized  rather  £than]  assessm  -nts  made  by  a 
limited  number  of  staff  personnel." 

“Our  communications  wi  th  the  product  division  is  Csic]  very 
good  now.  The  new  policy  may  work  if  the  product  division  players 
are  not  changed.  If  so,  the  communications  channels  will  remain 
the  same  £as  they  are  now]." 

“I  haven't  found  the  development  planners  particularly  in- 
formative on  what  the  real  needs  are  in  the  product  divisions.” 

Assessment  policy  will  improve  the  dialogue  provided  the 
product  divisions  do  not  become  unreasonably  adament  in  their  stand. 

It  will  raise  the  level  of  dialogue.  In  the  past  the  other 
side  said  I don't  want  it.  Now  they  can’t  just  say  I don't  want  it; 
they  have  to  tell  why. 

"The  policy  has  forced  product  divisions  to  review  potential 
sources  for  emerging  technologies  within  the  lab.  The  next  time  they 
perceive  a technology  void,  before  they  go  directly  to  industry  for  j 

solutions,  they  may  remember  a source  uncovered  during  an  assessment  | 

review. " j 

* 

"Good  to  get  all  levels  of  players  to  agree  on  criteria  for  i 

judging  programs.”  J 

"I  indicated  yes  only  because  the  program  may  be  seen  by  ■ 

more  people,  but  not  more  agencies." 

"There  should  be  extensive  dialogue  with  the  product  divi-  i 

sion,  but  I disagree  with  the  current  assessment  policy." 

It  £the  assessment  process]  has  served  to  make  everyone  de-  ? 
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QUESTION  14 

PRODUCT  DIVISION  PERSONNEL  COMMENTS 


I think  assessment  adds  a horizontal  communication  link. 
There  has  always  been  a good  funding  link  between  the  labs  and 
AFSC  (DL),  but  now  the  assessment  adds  the  link  to  AFSC  (XR).  It 
is  weak,  but  it  is  there. 

It  did  get  more  people  involved  with  reviewing  the  ADP. 

"[it]  creates  a formal  organizational  awareness,  and  that 
fosters  dialogue." 

The  assessment  policy  will  force  lab  people  to  talk  more 
openly.  It  will  force  them  to  make  estimates  for  the  future.  I 
think  it  will  increase  communications  within  the  product  division 
also. 


It  increases  the  arguments.  Some  lab  people  hate  our  guts 
right  now. 

I think  it  depends  on  the  situation.  One  guy  might  do  the 
whole  assessment. 

One  thing  you  have  to  recognize  is  how  these  jobs  are 
handled  once  they  come  in.  The  boss  looks  around  and  finds  some- 
one who  is  knowledgeable  in  that  technical  area  and  who  isn't  too 
busy  to  do  the  job.  The  next  time  the  program  gets  assessed  the 
guy  who  did  it  the  first  time  is  probably  busy  on  something  else  or 
has  been  transferred  out,  and  someone  else  is  assigned.  This  new  guy 
does  his  assessment  without  necessarily  having  the  knowledge  of  what 
the  first  guy  said,  and  his  opinion  might  be  different. 

You  have  to  consider  how  assessments  are  likely  to  be  made. 

A person  who  isn't  busy  will  get  the  job  and  will  learn  about  che 
program.  Most  likely  the  results  will  not  be  gotten  across  to  the 
engineers.  If  you  were  to  get  the  word  to  everyone  who  needed  it, 
the  labs  would  spend  all  of  their  time  educating  people.  It  is 
really  up  to  the  individual  engineer  to  keep  track  of  things  that 
happen  in  his  own  area  of  interest.  I think  it  is  the  informal 
system  that  gets  things  accomplished. 
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QUESTION  15 

LABORATORY  PERSONNEL  COMMENTS 


"The  technical  program  schedule  options  [are]  easy,  but 
schedules  of  probable  applications  appear  to  change  continuously.” 

"Payoff  is  program  related — as  is  ease  of  estimating  a 
schedule  that  can  be  met  within  dollars  provided." 

The  question  is  not  whether  the  product  divisions  have 
enough  time  and  manpower  to  devote  to  the  assessment  process.  Rather 
it  is  will  they  treat  it  as  important  with  respect  to  the  other  things 
they  have  to  do. 

I don't  know  how  the  product  division  intends  to  implement 
the  policy. 

The  assessment  policy  is  a waste  of  manpower. 
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QUESTION  16 

PRODUCT  DIVISION  PERSONNEL  COMMENTS 


There  are  two  kinds  of  costs  we  can  assess.  Life  cycle 
cost  is  virtually  impossible.  The  cost  of  ADP  itself  is  somewhat 
difficult. 


Cost  estimates  are  really  a problem  to  assess.  I went  to 
our  cost  people  when  making  an  assessment,  and  they  wouldn't  touch  it. 

What's  the  measurement  of  payoff?  Whit's  the  bottom  line? 
Many  times  payoffs  are  negated  by  problems  that  arise  during  the 
program. 


We  could  have  spent  more  time  on  payoff.  To  really  assess 
payoff  you  have  to  do  a detailed  systems  analysis  and  try  to  deter- 
mine how  the  system  performs  with  existing  technology  and  how  H 
would  perform  with  the  new  technology.  How  do  you  really  me.  re  the 
effect  of  a new  technology  on  combat  effectiveness? 

"Yes,  if  management  supplies  proper  priority." 

You  can  never  get  a handle  on  the  assessment  in  the  way  they 
would  like  for  you  to  do  it.  You  just  have  to  do  the  best  you  can. 

I think  that  the  assessment  process  is  valuable.  We  should 
do  it  in  preference  to  some  of  the  ’’mickey  mouse"  exercises  we  have 
to  do  now.  We  do  have  deadlines,  and  we  really  can’t  do  as  good  a 
job  as  you  ought  to  be  allowed. 

"The  answer  to  16a  is  contingent  on  having  sufficient  time 
and  manpower  available.  I do  not  think  we  now  have  either.” 

"Although  the  product  divisions  are  manpower  short,  every 
effort  should  be  given  to  assessing  both  old  and  new  start  ADPs  on 
a semi-annual  basis.  These  assessments  could  be  done  in  conjunc- 
tion with  and  by  those  people  engaged  in  system  studies  that  relate 
to  this  ADP." 

"Reconciliation  of  opposing  points  of  view  within  a product 
division  takes  the  most  time  and  effort.  The  tasks  themselves  may 
be  difficult,  but  these  are  easily  resolved  when  compared  to  the 
reconciliation  of  a product  division  assessment." 

Time  has  to  be  made  available.  We  can’t  afford  not  to  take 
the  time  to  do  the  assessments  properly,  I think  we  had  problems  the 
first  time  through  but  the  process  will  become  easier  with  time. 
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QUESTION  16 

OTHER  PERSONNEL  COJMENTS 


The  product  divisions  should  make  the  time  available  for 
the  assessment  process.  It’s  an  important  aspect  of  their  general 
responsibility. 
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QUESTION  17 

LABORATORY  PERSONNEL  COMMENTS 


An  assessment  means  an  evaluation  of  something  that  someone 
has  done.  The  problem  is  how  do  you  define  what  is  to  be  done.  We 
feel  like  we  are  going  out  and  culling  the  world  for  problems.  What 
are  the  product  divisions  doing  to  identify  problems  that  advanced 
development  programs  can  be  built  around?  The  product  division  has 
to  identify  what  to  be  responsive  to.  It's  hard  to  be  responsive 
to  a vacuum. 

You  get  results  from  ADTPs  as  much  as  ten  years  in  the  future. 
The  product  division  has  to  look  ahead  to  when  these  values  are  recog- 
nized and  not  look  to  today's  problems. 
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QUESTION  17 

PRODUCT  DIVISION  PERSONNEL  COMMENTS 


The  real  way  to  make  the  laboratories  responsive  is  to  | 

latch  on  to  the  right  person  at  the  laboratory.  If  he  is  inter-  | 

ested  in  what  you  are  trying  to  do,  the  program  will  sell.  | 

I 

You  should  have  had  a "before.”  Before,  they  were  completely  | 

nonresponsive.  J 

1, 

Essentially  no  change  in  lab  response.  fj 

J 

$ 

"A.  The  labs  are  not  responsive  to  product  division  needs.  | 

B.  This  deficiency  is  due,  in  part,  to  the  sterile  approach  | 

taken  by  the  labs  to  develop  their  new  programs.  They  do  not  seek  'i 

nor  apparently  want  product  division  inputs  to  shape  their  efforts.  | 

C.  The  basis  for  any  technology  transfer  is  the  existence  1 

of  an  adequate  technology  base.  By  a technology  base  is  meant  a f 

body  of  knowledge,  data  point  values,  curves,  etc.  that  depicts  how  | 

the  variables  affecting  a problem  are  interrelated  (e.g.,  the  old  f 

NACA  reports  of  lift  coefficients,  etc.)."  I 
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QUESTION  18 

LABORATORY  PERSONNEL  COMMENTS 


o 


The  process  will  be  all  right  after  it  is  going  for  some 
time.  It  was  a real  problem  in  bringing  the  technical  people  in  the 
product  division  up  to  speed  on  what  we  were  doing.  The  process  is 
also  going  to  help  in  getting  the  engineering  people  to  talk  to  the 
development  planning  people  in  the  product  divisions. 

"In  principle  the  product  division  assessment  of  advanced 
development  programs  is  good,  but  a waiting,  Qnd-using  system  can 
never  be  identified  for  all  advanced  development  programs.  Only 
after  demonstration  of  attainable  capabilities  combined  with  cost 
estimates  will  it  be  possible  to  sort  out  acceptable  using  weapon 
systems." 


One  of  the  best  things  I know  of  about  the  assessment  policy 
is  that  it  forces  specific  comments  from  the  product  divisions.  They 
no  longer  can  say  that  they  just  don't  like  it. 

"The  idea  is  good — the  implementation  is  questionable." 

"The  ADTP  assessment  policy  is  a good  idea,  but  the  imple- 
mentation is  critical  (i.e.,  timeliness  of  inputs,  etc.)" 

A key  thing  in  making  the  assessment  policy  work  is  the 
granularity  of  the  product  division  assessment.  The  program  element 
level  is  generally  a sufficient  level  for  the  assessment.  They 
should  not  nitpick  our  programs  or  attempt  to  be  project  engineers. 
Another  important  thing  is  that  a product  division  policy  must  be 
known  early  in  the  game.  There  has  never  been  a corporate  spokes- 
man for  the  product  division.  Perhaps  the  assessment  process  will 
generate  one. 

"I  agree  that  the  assessment  policy  is  good  insofar  as  it 
increases  the  likelihood  of  an  ADP  finding  a home  and  application 
in  the  product  divisions.  I see  a danger  that  overemphasis  on 
pleasing  a product  division  in  the  short  term,  which  is  where 
their  interest  always  is,  may  cause  the  long  term  technology  to 
suffer." 


It  would  be  a good  idea  if  it  was  worked  correctly.  I 
think  it  is  a rather  biased  situation  right  now. 

You  have  to  be  open-minded.  You  have  to  be  able  to  talk 
about  potential  applications  and  merit.  Recommendations  have  to 
be  made  on  potential  applications  and  merit.  If  the  product  divi- 
sions are  asked  to  give  a yes  or  no  answer,  you  will  get  a lot  of 
noes. 


I 

i 

* 
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1 


| 

You  don't  sell  a program  by  writing  a plan.  First  you  sell 
it,  then  you  write  the  plan.  I think  the  lab  should  work  closely  j 

with  the  product  divisions  in  the  planning  of  an  ADTP,  and  I think  jj 

the  product  division  should  h ve  a vote  regarding  an  ADTP,  but  I r- 

don't  think  they  should  have  a veto.  The  assessment  policy  gives  | 

them  a veto.  The  assessments  have  been  too  short  term;  I think  the  jj 

process  will  be  eliminated.  Air  Staff  disagrees  with  XR  having  that  | 

much  power.  They  have  already  turned  \round  one  assessment.  1 


I think  it  is  just  a way  to  make  money  decisions  for  people 
at  AFSC.  They  are  getting  somebody  to  do  their  job  for  them. 


The  likely  outcome  is  that  the  assessment  policy  will  do 

nothing . 
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The  ADTPs  were  almost  autonomous  within  the  labs.  The  seed 
was  planted  there,  and  the  benefits  were  accrued  and  received  there. 

I think  that  the  assessment  is  a good  policy  as  implemented. 
There  are  a few  snags,  a few  things  to  learn,  and  some  personality 
and  political  problems  to  work  out,  but  the  labs  are  forced  to  take 
some  hard  looks  at  what  they  are  doing.  Previously  we  went  to  pro- 
gram review  briefings,  and  we  made  comments  on  the  programs  that 
were  briefed.  The  feeling  here  was  that  these  efforts  had  no  real 
impact  on  what  the  labs  were  doing  and  that  it  was  a waste  of  time. 

We  have  super  internal  coordination  problems.  I rewrote  my 
assessment  at  least  six  times.  Both  groups  have  to  take  responsibility. 
The  product  divisions  have  the  authority  to  kill  a program  now.  What 
happens  if  a good  program  is  cancelled  because  of  a negative  product 
division  assessment. 

ADTP  assessments  are  a good  idea,  if  it  doesn't  get  down  to 
just  being  a rubber  stamp.  It  is  up  to  the  management  to  make  it 
work. 

"The  ADP  managers  appear  to  want  to  minimize  the  influence 
of  outside  (product  divisions)  sources  once  the  ADP  has  started. 

They  don't  return  phone  calls.  No  one  likes  to  be  exposed  and  re- 
exposed to  criticism." 

I don't  understand  how  the  assessments  are  used  once  they 
are  sent  to  AFSC.  I am  not  sure  I have  detected  how  it  is  going  to 
be  different. 

One  thing  that  bothers  me  about  the  assessment  process  is 
that  you  need  a balence  between  technology  for  technology's  sake  and 
technology  for  systems.  I fear  there  will  be  a tendency  to  overempha- 
size technology  for  systems. 


3 


2 

3 


GSM/SM/76S-21 


QUESTION  19 

LABORATORY  PERSONNEL  COMMENTS 


"The  amount  of  technology  transferred  involves  more  than 
just  specific  advanced  development  programs.  Much  technology  is 
transferred  through  increasing  industry  technical  capability — which 
frequently  is  factored  into  already  approved  or  even  ongoing  pro- 
duction programs.  In  addition,  6.4  budget  ceilings  frequently 
limit  the  number  of  ADPs  transferred  to  engineering  development." 

"With  regard  to  establishing  a technology  transfer  agency 
I must  emphasize  professional  type  technology  transfer  individuals; 
otherwise  it  would  be  a detriment." 

"Regarding  5 above,  no  new  bureacracy,  but  exploration  of 
an  agency  such  as  product  division  plans." 
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QUESTION  19 

PRODUCT  DIVISION  PERSONNEL  COMMENTS 


"[The]  problem  with  these  ideas  is  that  they  ’force'  the 
transfer  of  technology;  however,  it  is  important  to  identify  tech- 
nology that  shouldn't  be  transferred!"  A valid  output  of  an  ADTP 
is  that  this  technology  is  no  good. 

The  first  three  items  would  do  nothing  but  lead  to  a hugh 
"pissing  contest."  Number  5 works  best  informally.  The  product 
division  engineer  needs  to  get  involved  with  a program  which  is  of 
interest  to  him  and  work  with  the  lab  people  on  a daily  basis  if 
necessary. 

You  would  have  to  somehow  hold  people  accountable  for  the 
ratings.  They  can't  just  be  open  loop. 

Number  4 sounds  pretty  bad  to  me.  It  doesn’t  allow  the  labs 
to  turn  around.  The  product  divisions  should  have  a voice  in  how 
the  funds  are  spent,  but  should  not  control  the  funds. 

The  idea  of  making  future  funding  contingent  on  past  trans- 
fers is  good  only  if  it  is  a basis  for  reward.  The  labs  shouldn't 
be  punished  for  not  transferring  a technology  because  there  are  many 
things  that  affect  whether  or  not  a system  will  use  the  results  of 
a particular  advanced  development  program. 

I hate  to  call  it  a transfer  agency.  The  job  should  be 
done  by  the  development  planning  people;  except  that  right  now  I 
think  we  are  dominated  by  the  rest  of  the  product  division. 

The  labs  should  rate  the  content  of  what  they  put  out  in- 
stead of  concentrating  on  just  distributing  the  stuff.  You  don't 
need  another  level  of  bureacracy.  If  the  job  is  done  properly,  the 
technology  will  sell  itself. 

A technology  transfer  agency  sounds  good  in  theory,  but  a 
typical  Air  Force  reaction  to  a problem  is  to  start  an  agency  with 
experts  and  let  them  come  up  with  answers.  I haven't  seen  it  work 
very  well  in  practice. 

The  basic  concept  of  a technology  cransfer  agency  is  a 
beautiful  theory,  but  it  wouldn't  work  in  practice.  I think  it 
would  be  anothei  drain  on  manpower  and  would  accomplish  nothing. 

The  technology  transfer  theory  is  that  somebody  looks  at  the  future 
and  decides  what  programs  the  labs  should  pursue.  The  labs  do  their 
thing  and  put  the  results  on  the  shelf.  Then  the  requirement  comes 
along  and  uses  it.  In  practice  though  it  doesn't  work  like  this. 

The  future  is  not  what  we  think  it's  going  to  be,  and  by  the  time  we 
see  what  we  would  like  to  do  we  rush  it  through  with  the  SPOs  prime 
contractor  doing  the  job.  This  normally  bypasses  the  lab  because 
it  takes  them  so  long  to  get  a program  started. 
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Explanation  of  Tables 

The  tables  in  this  appendix  appresent  the  values  computed  for  the 
key  parameters  of  the  coefficient  of  concordance  test.  The  responses  to 
Questions  6,  7,  8,  11,  12,  15a,  16a,  and  19  were  analyzed  in  this  manner. 
The  entries  in  the  first  two  table  columns  are  a description  of  the  param- 
eters contained  in  the  various  table  row.  Each  of  the  remaining  table 
columns  represent  values  of  this  parameter  computed  for  the  responses 
given  by  personnel  in  the  demographic  grouping  described  by  the  table 
column  heading. 

The  first  five  (eight  for  Questions  7 and  8)  rows  list  the  means 
of  the  ranks  and  the  means  of  the  response  computed  for  each  of  the  factors 
ranked.  The  values  have  been  rounded.  The  remaining  rows  list  the  value 
of  several  parameters  computed  in  performing  the  test.  Note  that  the  sum 
of  the  ranks  assigned  to  a particular  factor  (R^)  can  be  computed  by  mul- 
tiplying the  mean  rank  times  the  number  of  respondents,  denoted  as  n in 
the  tables. 

The  following  computational  forms  were  used: 

Friedman  Test  Statistic  (T) 

T - (*-D$ 

3* 
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nk(kx-/)  ~ 
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number  of  factor.*?  ranked. 


. th 


sum  of  ranks  assigned  to  the  j — factor. 


n 

t 


number  of  personnel  interviewed  within  any  given  demo- 
graph  is  grouping. 


number  of  tied  ranks  assigned  by  a person  interviewed 
per  group  of  tied  ranks. 


(Gibbons,  1971:233,  247,  249,  257) 
Kendall's  Coefficient  of  Concordance  (W) 


$ 


(Gibbons,  1971:257) 


Degrees  of  Freedom  (v1}V2) 


V,*  A-/  - */n 

\)^-(n  -/ ) V/  (Gibbons,  1971:255) 

Note:  The  values  tabulated  have  been  rounded. 

F Statistic  (F) 
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TABLE  F-I 

RESULTS  OF  COEFFICIENT  OF  CONCORDANCE  TESTS  (QUESTION  6) 


QUESTION  6s  Please  indicate  the  extent  that  you  agree  or  disagree  that 
the  ADTP  assessment  policy  will  encourage  tht  occurrence 
of  such  events. 


Respondents 

All 

Planning  Activi- 
ties Which  May 
Expose  Problems 

Mean  Rank 
Mean  Response 

3.01 

5.26 

Provide  Forum 
For  Problem 
Resolution 

Mean  Rank 
Mean  Response 

2.45 

4.82 

Strong  Product 
Division  Support 

Mean  Rank 
Mean  Response 

3.04 

5.29 

Establish  Tech- 
nology Is  Mature 

Mean  Rank 
Mean  Response 

2.66 

4.95 

Establish  Early 

Informational 

Linkage 

Mean  Rank 
Mean  Response 

3.84 

5.95 

n 

38 

Friedman  Tost  Statistic 

21.8 

Kendall's  Coefficient 
of  Concordance  (W> 

.144 

Degrees  of  Freedom 

(Vi;v3) 

4jl46 

F Statistic 

6.20 

Level  of  Significance  (a) 
(1.2-4  = 1.2  x 10"\  etc.) 

1.2"' 

Lab 

P.D. 

Lab 

A 

Lab 

B 

P.D. 

A 

P.D. 

B 

2.88 

4.71 

3.00 

5.82 

3.08 

4.17 

3,00 

5,33 

3.13 

6.13 

2.89 

5.56 

2.24 

4.24 

2.68 

5.53 

2.75 

4.00 

1,72 

4,44 

3.31 

6.13 

2.11 

5.00 

3.44 

5.35 

2.82 

5.41 

2.58 

4.00 

3.83 

6.00 

2.63 

5.38 

3.00 

5.44 

2.68 

4.53 

2.56 

5,29 

2.50 

3.83 

2.72 

5.11 

2.19 

5.75 

2.89 

4.89 

3.77 

5.59 

3.94 

6.41 

4.08 

5.00 

3.72 

5.89 

3.75 

6.50 

4.11 

6.33 

17 

17 

6 

9 

8 

9 

13.0 

10.6 

5.16 

13.5 

6.25 

9.16 

.192 

.155 

.215 

,376 

.195 

.254 

4;62 

4;  62 

4;18 

4;  30 

4;2  6 

4;  30 

3.79 

2.95 

1.37 

4.82 

1.70 

2.73 

8.1"3 

2.7"2 

2.8"1 

4.0"3 

1.8"1 

4.8" 
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TABLE  F-I  (CONTINUED) 


QUESTION  6j  Please  indicate  the  extent  that  you  agree  or  disagree  that 
the  ADTP  assessment  policy  will  encourage  the  occurrence 
of  such  events. 


i 


Respondents 

GS  15 
LC  & 
Above 

GS  14 
Maj  & 
Below 

Over 

20 

Years 

Under 

20 

Years 

More 
Than 
2 Yrs 

Less 
Than 
2 Yrs 

Planning  Activi- 
ties Which  May 
Expose  Problems 

Mean 

Mean 

Rank 

Response 

2.42 

5.00 

3.26 

5.43 

3.16 

5.42 

2.67 

5.07 

3.36 

5.52 

2.27 

4.85 

Provide  Forum 
For  Problem 
Resolution 

M.>an 

Mean 

Rank 

Response 

2.23 

5.00 

2.60 

4.81 

2.45 

5.00 

2.47 

4.73 

2.69 

5.05 

2.08 

4.62 

Strong  Product 
Division  Support 

Mean 

Mean 

Rank 

Response 

3.46 

5.69 

2.93 

5.19 

2.71 

4.95 

3.67 

5.93 

2.61 

4.95 

3.96 

6.08 

Establish  Tech- 
nology Is  Mature 

Mean 

Mean 

Rank 

Response 

2.85 

5.31 

2.48 

4.67 

2.66 

5.11 

2.57 

4.67 

2.41 

4.91 

2.96 

4.92 

Establish  Early 

Informational 

Linkage 

Mean 

Mean 

Rank 

Response 

4.04 

6.15 

3.74 

5.91 

4.03 

6.00 

3.63 

6.00 

3.93 

6.00 

3.73 

6.00 

n 

13 

21 

19 

15 

21 

13 

Friedman  Test  Statistic 

14.7 

11.5 

15.5 

11.0 

17.8 

17.9 

Kendall's  Coefficient 
of  Concordance  (W) 

.283 

.137 

.203 

.189 

.212 

.34* 

Degrees  of  Freedom 
(Vj ; v2) 

4;46 

4j78 

4;70 

4;  54 

4{78 

4;46 

F Statistic 

4.73 

3.19 

4.60 

3.16 

5.38 

6.30 

Level  of  Significance  (a) 
(2.8'3  = 2.8  x 10"3,  etc.) 

2,8-3 

1.8"2 

2.3“3 

2.1-2 

7.0"4 

4.0~4 

I 
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TABLE  F-II 

RESULTS  OF  COEFFICIENT  OF  CONCORDANCE  TESTS  (QUESTION  7) 


QUESTION  7s  Before  the  ADTP  assessment  policy  went  into  effect,  the  fol- 
lowing factors  represented  a barrier  to  the  transfer  of  ADP 
technology  to  system  application. 


j Respondents 

All 

Lab 

P.D. 

Lab 

Lab 

P.D. 

P.D. 

f 

j 

A 

B 

A 

B 

1 

; Inertia 

Mean  Rank 

5.54 

6.12 

4.97 

6.67 

5.94 

4.94 

5.00 

Barriers 

j 

Mean  Response 

5.11 

5.35 

4.88 

5.33 

5.67 

4.75 

5.00 

‘ Lack  of  Incen- 

Mean  Rank 

4.07 

3.59 

4.77 

4.25 

3.06 

4.88 

4.67 

| tive  Structure 

Mean  Response 

4.18 

3.35 

5.18 

3.67 

3.22 

5.13 

5.22 

! 

i Cost 

Mean  Rank 

4.68 

5.00 

4.41 

4.83 

4.83 

3.75 

5.00 

j Barriers 

Mean  Response 

4.61 

4.24 

4.94 

4.00 

4.44 

4.25 

5.56 

i 

Communication 

Mean  Rank 

5.11 

4.77 

5.15 

4.75 

4.89 

4.69 

5.56 

Barriers 

Mean  Response 

4.92 

4.24 

5.47 

4.33 

4.56 

5.13 

5.78 

s 

i Time 

Mean  Rank 

5.17 

5.03 

5.50 

4.50 

5,89 

4.88 

6.06 

1 Barriers 

Mean  Response 

5.00 

4.41 

5.71 

4.17 

5,11 

5.38 

6.00 

t 

Geographic 

Mean  Rank 

2.55 

2.50 

2.53 

3.00 

2.39 

3.06 

2.06 

Distance 

Mean  Response 

3.03 

2.65 

3.24 

3.00 

2.67 

3.88 

2.67 

Transfer  Manage- 

Mean Rank 

5.24 

5.71 

4.94 

4.00 

6.39 

5.38 

4.56 

ment  Structure 

Mean  Response 

4.92 

5.00 

4.88 

3.33 

5.89 

5.25 

4.56 

Technology 

Mean  Rank 

3.65 

3.29 

3.74 

4.0C 

2.61 

4.44 

3.11 

Barriers 

Mean  Response 

3.68 

2.71 

4.41 

3.17 

2.44 

4.50 

4.33 

n 

38 

17 

17 

6 

9 

8 

9 

Friedman  Test  Statistic 

51.9 

35.0 

21.1 

9.35 

28.9 

6.02  ' 

21.1 

Kendall's  Coefficient 

| of  Concordance 

(W) 

.195 

.294 

.177 

.223 

.458 

.108 

.336 

| Degrees  of  Freedom  (VjjVg) 

7 5 256 

7;110 

7;110 

7;  32 

7;  54 

7;46 

7;  54 

* 

I F Statistic 

8.97 

6.66 

3.45 

1.43 

6,77 

,844 

4.04 

1 Level  of  Significance  (a) 

-9 

2.0  * 

1.4-6 

2.2'3 

2.3"1 

9.  O'  6 

5.6"1 

1.3“3 

(2.0~9  = 2.0  x 10-9,  etc.) 
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TABLE  F-II  (CONTINUED)  1 


QUESTION  7t  Before  the  ADTP  assessment  policy  went  into  effect,  the  fol- 
lowing factors  represented  a barrier  to  the  transfer  of  ADP 
technology  to  system  application. 


Respondents 

GS  15 

GS  14 

Over 

Under 

More 

Less 

LC  & 

Maj  & 

20 

20 

Than 

Than 

Above 

Below 

Years 

Years 

2 Yrs 

2 Yrs 

Inertia 

Mean  Rank 

5.89 

5.33 

6.05 

4.90 

5.83 

5.08 

Barriers 

Mean  Response 

5.31 

5.00 

5.63 

4.47 

5.52 

4.46 

Lack  of  Incen- 

Mean Rank 

3.77 

4.43 

3.87 

4.57 

4.41 

3.81 

tive  Structure 

Mean  Response 

4.00 

4.43 

4.16 

4.40 

4.57 

3.77 

Cost 

Mean  Rank 

4.62 

4.76 

4.66 

4.77 

4.31 

5.35 

Barriers 

Mean  Response 

4.46 

4.67 

4.84 

4.27 

4.57 

4.62 

Communication 

Mean  Rank 

4.73 

5.10 

4.32 

5.77 

4.98 

4.92 

Barriers 

Mean  Response 

4.54 

5.05 

4.58 

5.20 

5.14 

4.39 

Time 

Mean  Rank 

5.50 

5.12 

5.26 

5.27 

5.00 

5.69 

Barriers 

Mean  Response 

4.92 

5.14 

5.16 

4.93 

5.19 

4.85 

Geographic 

Mean  Lank 

2.39 

2.60 

2.79 

2.17 

2.71 

2.19 

Distance 

Mean  Response 

3.00 

2.91 

3.37 

2.40 

3.33 

2.31 

Transfer  Manage 

- Mean  Rank 

5.31 

5.33 

5.26 

5.40 

4.91 

6.00 

raent  Structure 

Mean  Response 

4.77 

5.05 

4.90 

5.00 

4.81 

5.15 

Technology 

Mean  Rank 

3.81 

3.33 

3.79 

3.17 

3.86 

2.96 

Barriers 

Mean  Response 

3.85 

3.38 

4.00 

3.00 

4.00 

2.85 

n 

13 

21 

19 

15 

21 

13 

Friedman  Test  Statistic 

21.7 

29.6 

27.7 

29.2 

24.8 

31.6 

Kendall's  Coefficient 

of  Concordance 

(W) 

.239 

.202 

.208 

.278 

.169 

.347 

Degrees  of  Free 

doro  (vx;vs) 

7 ; 82 

7}  138 

7}  124 

7}  96 

7}  138 

7 5 82 

F Statistic 

3.77 

5.05 

4.74 

5.38 

4.06 

6.37 

Level  of  Significance  (&) 

(1.4~3  = 1.4  x 

10  3,  etc.) 

1.4-3 

4.  O'5 

9.6~5 

3.2~5 

4.5“4 

5.5-6 
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TABLE  F-III 

RESULTS  OF  COEFFICIENT  OF  CONCORDANCE  TESTS  (QUESTION  8) 


QUESTION  8t  Now  that  the  ADTP  assessment  policy  is  in  effect,  the  following 
factors  will  represent  (or  "still  represent")  a barrier  to  the 
transfer  of  ADP  technology  to  system  application. 


Respondents 

All 

Lab 

P.D. 

Lab 

Lab 

P.D. 

P.D. 

A 

B 

A 

B 

Inertia 

Mean  Rank 

5.82 

6.62 

5.27 

7.00 

6.50 

5.44 

5.11 

Barriers 

Mean  Response 

4.42 

4.88 

4.24 

5.17 

4.89 

4.38 

4.11 

Lack  of  Incen- 

Mean Rank 

4.55 

3.82 

5.24 

3.92 

3.83 

4.88 

5.56 

tive  Structure 

Mean  Response 

3.61 

2.94 

4.47 

3.17 

2.89 

4.13 

4.78 

Cost 

Mean  Rank 

5.26 

4.74 

5.59 

4.33 

4.61 

4.94 

6.17 

Barriers 

Mean  Response 

4.05 

3.53 

4.59 

3.50 

3.44 

3.88 

5.22 

Communication 

Mean  Rank 

3.59 

3.59 

3.71 

4.67 

2.89 

3.94 

3.50 

Barriers 

Mean  Response 

3.29 

3.00 

3.58 

3.83 

2.67 

3.75 

3.44 

Time 

Mean  Rank 

5.53 

5.85 

5.71 

5.50 

6.72 

5.13 

6.22 

Barriers 

Mean  Response 

4.24 

4.24 

4.82 

4.17 

4.78 

4.38 

5.22 

Geographic 

Mean  Rank 

3.29 

3.29 

3.12 

4.00 

3.16 

4.19 

2.17 

Distance 

Mean  Response 

2.87 

2.56 

3.12 

3.17 

2.44 

3.88 

2.44 

Transfer  Manage- 

Mean Rank 

4.26 

4.62 

3.97 

3.33 

5.05 

3.56 

4.33 

ment  Structure 

Mean  Response 

3.47 

3.59 

3.47 

2.67 

4.11 

3.00 

3.89 

Technology 

Mean  Rank 

3.70 

3.41 

3.41 

3.25 

3.22 

3.94 

2.94 

Barriers 

Mean  Response 

2.90 

2.41 

3.18 

2.50 

2.33 

3.13 

3.22 

n 

38 

17 

17 

6 

9 

8 

9 

Friedman  Test  Statistic 

49.3 

33.7 

28.8 

13.8 

26.0 

6.2 

29.0 

Kendall's  Coefficient 

of  Concordance  (W) 

.185 

.283 

.242 

.329 

.413 

.111 

.461 

Degrees  of  Freedom  (VjjVg)  7;256  7;110  7;110  7;32  7 ; 54  7;46  7; 54 
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TABLE  F-III  (CONTINUED) 


QUESTION  8:  Now  that  the  ADTP  assessment  policy  is  in  effect,  the  following 

factors  will  represent  (or  "still  represent")  a barrier  to  the 
transfer  of  ADP  technology  to  system  application. 


Respondents 

GS  15 
LC  & 
Above 

GS  14 
Maj  & 
Below 

Over 

20 

Years 

Under 

20 

Years 

More 
Than 
2 Yrs 

Less 
Than 
2 Yrs 

Inertia 

Barriers 

Mean  Rank 
Mean  Response 

6.81 

4.92 

5.41 

4.33 

6.08 

4.79 

5.77 

4.27 

6.02 

4.81 

5.81 

4.15 

Lack  of  Incen- 
tive Structure 

Mean  Rank 
Mean  Response 

4.27 

3.39 

4.74 

3.91 

4.16 

3.63 

5.07 

3.80 

4.38 

3.81 

4.85 

3.54 

Cost 

Barriers 

Mean  Rank 
Mean  Response 

5.39 

4.00 

5.02 

4.10 

5.42 

4.47 

4.83 

3.53 

5.14 

4.19 

5.19 

3.85 

Communication 

Barriers 

Mean  Rank 
Mean  Response 

3.54 

3.00 

3.71 

3.48 

3.24 

3.16 

4.17 

3.47 

3.79 

3.52 

3.42 

2.92 

Time 

Barriers 

Mean  Rank 
Mean  Response 

5.39 

4.00 

6.02 

4.86 

5.82 

4.68 

5.73 

4.33 

5,52 

4.62 

6.19 

4.39 

Geographic 

Distance 

Mean  Rank 
Mean  Response 

2.96 

2.77 

3.36 

2.91 

3.42 

3.21 

2.93 

2.40 

3.47 

3.24 

2.77 

2.23 

Transfer  Manage 
ment  Structure 

- Mean  Rank 
Mean  Response 

4.31 

3.62 

4.29 

3.48 

4.47 

4.00 

4.10 

2.93 

4.17 

3.52 

4.50 

3.54 

Technology 

Barriers 

Mean  Rank 
Mean  Response 

3.35 

2.71 

3.45 

2.81 

3.42 

3.05 

3.40 

2.47 

3.50 

2.91 

3.27 

2.62 

n 

13 

21 

19 

15 

21 

13 

Friedman  Test  Statistic 

29.6 

28.0 

34.6 

23.9 

28.7 

27.6 

Kendall's  Coefficient 
of  Concordance  (W) 

.326 

.197 

.260 

.227 

.195 

.303 

Degrees  of  Freedom  (v^Vg) 

7;  82 

7 5 138 

7}  124 

7;  96 

7 *,138 

7;  82 

F Statistic 

5.80 

4.91 

6.33 

4.12 

4.86 

5.22 

Level  of  Significance  (a) 
(1.8**5  = 1.8  x 10~5,  etc.) 

1.8'5 

5.6"5 

2.3~6 

5.3'A 

6.4"5 

5.9~5 
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TABLE  F-IV 

RESULTS  OF  COEFFICIENT  OF  CONCORDANCE  TESTS  (QUESTION  11) 


Question  lit  What  is  your  best  estimate  of  the  percent  of  your  working 
time  that  was  spent  communicating  with  personnel  in  the 
agencies  listed  below  regarding  new  ADPs  before  the  ADTF 
assessment  requirement  went  into  effect? 


Respondents 

All 

Lab 

P.D. 

Lab 

A 

Lab 

B 

P.D. 

A 

P.D. 

B 

Higher 

Headquarters 

Mean  Rank 
Mean  Response 

3.42 

7.86 

4.06 

11.88 

2.77 

3.12 

4.25 

13.33 

3.83 

10.56 

3.06 

5.63 

2.50 

.89 

Product 

Divisions 

Mean  Rank 
Mean  Response 

3.33 

7.50 

3.38 

5.38 

3.24 

9.94 

2.92 

5.67 

3 . '2 
5.22 

2.V.8 

10.63 

3.56 

9.33 

Laboratories 

Mean  Rank 
Mean  Response 

3.78 

9.00 

3.22 

8.19 

4.27 

3.12 

3.50 

11.33 

2.94 

5.89 

4.31 

13.13 

4.22 

3.67 

AFLC 

Mean  Rank 
Mean  Response 

1.93 

.89 

1.56 

.81 

2.32 

1.06 

1.92 

.50 

1.33 

.89 

2.31 

1.25 

2.33 

.89 

Operational 

Commands 

Mean  Rank 
Mean  Response 

2.54 

3.00 

2.78 

5.31 

2.41 

1.18 

2.42 

3.83 

3.17 

6.67 

2.44 

1.88 

2.39 

.56 

n 

36 

16 

17 

6 

9 

8 

9 

Friedman  Test 

Statistic 

43.3 

25.5 

27.8 

10.8 

15.5 

13.6 

16.3 

Kendall's  Coefficient 
of  Concordance  (W) 

.301 

.398 

.409 

.449 

.431 

.426 

.453 

Degrees  of  Freedom 
(VlJV2) 

4;  138 

4;  58 

4;62 

4, 18 

4;  30 

4;26 

4;  30 

F Statistic 

15.06 

9.91 

11.07 

4.08 

6.07 

5.20 

6.63 

Level  of  Signif icanc e (a) 
O.r10  = 3.1  x 10"L0,  etc.) 

3.r10  3.5~6 

7.7" 

? 1.6-: 

! x.r3 

; 3.3~3 

6.0_ 
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TABLE  F-IV  (CONTINUED) 


Question  lit  What  is  your  best  estimate  of  the  percent  of  your  working 
time  that  was  spent  communicating  with  personnel  in  the 
agencies  listed  below  regarding  new  ADPs  before  the  ADTP 
assessment  requirement  went  into  effect? 


Respondents 

GS  15 
LC  & 
Above 

GS  14 
Maj  & 
Below 

Over 

20 

Years 

Under 

20 

Years 

More 
Than 
2 Yrs 

Less 
Than 
2 Yrs 

Higher 

Headquarters 

Mean  Rank 
Mean  Response 

2.96 

3.15 

3.68 

10.10 

3.11 

5.21 

3.79 

10.29 

3.21 

6.10 

3.71 

9.58 

Product 

Divisions 

Mean  Rank 
Mean  Response 

3.54 

2.15 

3.15 

11.35 

3.40 

5.90 

3.18 

10.21 

3.24 

9.62 

3.42 

4.42 

Laboratories 

Mean  Rank 
Mean  Response 

4.19 

4.62 

3.48 

10.45 

3.97 

8.42 

3.46 

7.79 

4.07 

10.10 

3.21 

4.75 

AFLC 

Mean  Rank 
Mean  Response 

1.85 

.46 

2.03 

1.25 

2.13 

1.16 

1.71 

.64 

2.14 

1.10 

1.63 

.67 

Operational 

Commands 

Mean  Rank 
Mean  Response 

2.46 

1.08 

2.68 

4.55 

2.40 

1.21 

2.86 

5.86 

2.33 

1.38 

3.04 

6.33 

n 

13 

20 

19 

14 

21 

12 

Friedman  Test  S 

tatistic 

24.1 

18.9 

25.2 

17.4 

30.5 

14.75 

Kendall’s  Coefficient 
of  Concordance  (W) 

.464 

.236 

.332 

.310 

.363 

.307 

Degrees  of  Freedom 
(v.;v2) 

4;46 

4;74 

4;70 

4;  50 

4;78 

4;42 

F Statistic 

10.37 

5.88 

8.94 

5.85 

11.41 

4.88 

Level  of  Significance  (a) 
(4.5  b = 4.5  x 10  6,  etc.) 

4.5"6 

3.7~4 

6.8~6 

6.r« 

2.4'7 

2.5~3 
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TABLE  F-V 


RESULTS  OF  COEFFICIENT  OF  CONCORDANCE  TESTS  (QUESTION  12) 


Question  12t  What  is  your  best  estimate  of  the  percent  of  your  working 
time  that  is  or  will  be  spent  communicating  with  personnel 
in  the  agencies  listed  below  regarding  new  ADPs  now  that 
the  ADTP  assessment  requirement  is  in  effect? 


Respondents 

All 

Lab 

P.D. 

Lab 

A 

Lab 

B 

P.D. 

A 

P.D. 

B 

Higher 

Headquarters 

Mean  Rank 
Mean  Response 

3.21 

8.06 

3.69 

11.25 

2.71 

3.94 

4.08 

13.83 

3.33 

9.11 

2.69 

6.25 

2.72 

1.89 

Product 

Divisions 

Mean  Rank 
Mean  Response 

3.56 

8.67 

3.88 

7.44 

3.24 

10.18 

3.58 

7.67 

4.17 

7.00 

3.19 

10.13 

3.28 

10.22 

Laboratories 

Mean  Rank 
Mean  Response 

3.96 

10.17 

3.13 

8.44 

4.77 

9.77 

3.17 

11.50 

2.94 

5,67 

4.81 

13.00 

4.72 

6.89 

AFLC 

Mean  Rank 
Mean  Response 

1.81 

1.17 

1.53 

1.13 

2.12 

1.35 

1.83 

1,00 

1.33 

1.11 

2.00 

1.38 

2.22 

1.33 

Operational 

Commands 

Mean  Rank 
Mean  Response 

2.47 

4.14 

2.78 

6.88 

2.18 

1.82 

2.33 

7.50 

3.22 

7,00 

2.31 

2.75 

2.06 

1.00 

n 

36 

16 

17 

6 

9 

8 

9 

Friedman  Test 

Statistic 

50.2 

24.8 

39.8 

9.27 

17,0 

20.7 

19.6 

Kendall’s  Coefficient 
of  Concordance  (W) 

.349 

.387 

.585 

.386 

.472 

.647 

.543 

Degrees  of  Freedom 
<vi;v2) 

4;  138 

4;  58 

4;  62 

4;  18 

4;  30 

4;  26 

4;30 

F Statistic 

18.76 

9.47 

22.54 

3.15 

7.16  12.81 

9.51 

Level  of  Significance  (a) 
(2.5**12  = 2.5  x 10"12,  etc,) 

2.5-12  5.8_< 

5.  ,-11  -2 

1.6  4.0 

3.6^ 

‘ 6.9- 

5 4.3" 
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TABLE  F-V  (COKTll-JED) 


Question  12i  What  is  your  best  estimate  of  the  percent  of  your  working 
time  that  is  or  will  be  spent  communicating  with  personnel 
in  the  agencies  listed  below  regarding  new  ADPs  now  that 
the  ADTP  assessment  requirement  is  in  effect? 


Respondents 

GS  15 

GS  14 

Over 

Under 

More 

Less 

LC  & 

Maj  & 

20 

20 

Than 

Than 

Above 

Below 

Years 

Years 

2 Yrs 

2 Yrs 

Higher 

Mean  Rank 

3.00 

3.30 

3.13 

3.25 

3.10 

3.33 

Headquarters 

Mean  Response 

4.08 

9.70 

5.53 

10.14 

6.33 

9.50 

Product 

Mean  Rank 

3.71 

3.40 

3.63 

3.43 

3.60 

3.46 

< 

Divisions 

Mean  Response 

3.77 

12.15 

7.42 

10.79 

10.81 

5.42 

Laboratories 

Mean  Rank 

4.31 

3.75 

4.11 

3.79 

4.29 

3.42 

Mean  Response 

7.39 

10.25 

8.53 

9.93 

11.24 

5.42 

AFLC 

Mean  Rani 

1.73 

1.90 

1.95 

1.68 

1.88 

1.75 

Mean  Response 

.77 

1.55 

1.58 

.79 

1.48 

.83 

Operational 

Mean  Rank 

2.19 

2.65 

2.18 

2.86 

2.14 

3.04 

Commands 

Mean  Response 

1.77 

5.90 

1.79 

7.64 

1.95 

8.33 

n 

13 

20 

19 

14 

21 

12 

i 

f 

Friedman  Test  Statistic 

28.6 

20.0 

31.3 

16.8 

40.8 

11.08 

t! 

Kendall's  Coefficient 

li 

of  Concordance 

(W) 

.550 

.249 

.411 

.301 

.486 

.231 

* 

Degrees  of  Freedom 

r 

i 

(ViJVg) 

4;46 

4;74 

4;70 

4;  50 

4;78 

4;42 

\ 

F Statistic 

14.6 

6.31 

12.59 

59 

18.91 

3.30 

S 

Level  of  Significance  (a) 

] 

(9.0-8  = 9.0  x 

10  8,  etc.) 

9.0-8 

2.0~4 

9.0**8 

8.5**4 

6.7'11 

1.9**  ^ 

i 
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TABLE  F-VI 

RESULTS  OF  COEFFICIENT  OF  CONCORDANCE  TESTS  (QUESTION  15a) 


Question  15a*  If  you  work  in  a laboratory,  how  difficult  is  it  (or 
"would  it  be")  for  you  to  obtain  what  you  feel  is 
reliable  information  on  each  of  the  following  items 
to  include  in  the  technology  program  plan? 


Respondents 

Lab 

Lab 

A 

Lab 

B 

Payoff 

Mean  Rank 
Mean  Response 

3.53 

4.41 

4.17 

5.17 

3.50 

4.33 

Schedules 

Mean  Rank 
Mean  Response 

3.56 

4.41 

3.75 

4.67 

3.22 

4.00 

ADP  Cost  Estimates 

Mean  Rank 
Mean  Response 

2.71 

3.65 

3.00 

4.17 

2.28 

3.22 

Priority 

Mean  Rank 
Mean  Response 

2.15 

3.00 

1.92 

3.00 

2.28 

3.22 

Alternate  Approaches 

Mean  Rank 
Mean  Response 

3.06 

4.00 

2.17 

3.50 

3.72 

4.44 

n 

17 

6 

9 

Friedman  Test  Statistic 

10.9 

10.1 

6.7 

Kendall’s  Coefficient 
of  Concordance  (W) 

.160 

.421 

.212 

Degrees  of  Freedom 
(V1»V2> 

4;62 

4;18 

4;30 

F Statistic 

3.04 

3.64 

2.16 

Level  of  Significance  (a) 
(2.4  2 •=  2.4  x 10  2,  etc.) 

2.4~2 

?.4~2 

9.8" 

4 


i 

3 

•ii 


I 

i 
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TABLE  F-VII 

RESULTS  OF  COEFFICIENT  OF  CONCORDANCE  TESTS  (QUESTION  16a) 


Question  16a:  If  you  work  in  a product  division,  how  difficult  is 

it  (or  "would  it  be")  for  you  to  perform  what  you 
feel  is  a reliable  assessment  of  an  ADP  with  respect 
to  each  of  the  following  items? 


Respondents 

P.D. 

P.D. 

P.D. 

A 

B 

Payoff 

Mean  Rank 

3.38 

3.25 

3.50 

Mean  Response 

A. 18 

3.25 

5.00 

Schedules 

Mean  Rank 

2.97 

2.94 

3.00 

Mean  Response 

3.71 

3.13 

4.22 

ADP  Cost  Estimates 

Mean  Rank 

2.24 

1.88 

2.56 

Mean  Response 

3.24 

2.50 

3.89 

Priority 

Mean  Rank 

3.32 

3.56 

3.11 

Mean  Response 

3.94 

3.50 

4.33 

Decision  Options 

Mean  Rank 

3.09 

3.38 

2.83 

Mean  Response 

4.24 

3.88 

4.56 

n 

15 

8 

9 

Friedman  Test  Statistic 

7.3 

6.9 

2.4 

Kendall's  Coefficient 

of  Concordance  (W) 

.108 

.215 

.066 

Degrees  of  Freedom 

(ViJVg) 

4;  62 

4;  26 

4;  30 

F Statistic 

1.93 

1.92 

.561 

Level  of  Significance 

(a) 

(1.2-1  = 1.2  x 10-1,  i 

etc.) 

1.2_1 

1 

<r 

r—l 

6.9~ 
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TABLE  F-VIII 


RESULTS  OF  COEFFICIENT  OF  CONCORDANCE  TESTS  (QUESTION  19) 


Question  19s  Please  indicate  the  extent  that  you  agree  or  disagree  that 
these  ideas  have  enough  merit  to  warrant  further  consider- 
ation. 


Respondents 

Ail 

Lab 

P.D. 

Lab 

A 

Lab 

B 

P.D. 

A 

P.D. 

B 

Product  Divi- 
sions Rate  Labs 

Mean 

Mean 

Rank 

Response 

3.46 

4.24 

3.38 

3.94 

3.65 

4.82 

3.08 

3.67 

3,56 

3.67 

3.56 

4.50 

3.72 

5.11 

Labs  Rate  Pro- 
duct Divisions 

Mean 

Mean 

Rank 

Response 

3.18 

3.95 

3.00 

3.59 

3.41 

4.53 

3.08 

3.67 

3.00 

3.22 

3.13 

4.25 

3.67 

4.78 

Both  (1)  and  (2) 

Mean 

Mean 

Rank 

Response 

3.22 

4.03 

3.24 

3.77 

3.27 

4.53 

3.08 

3.67 

3.44 

3.56 

3.13 

4.25 

3.39 

4.78 

Make  Future 
Funding  Contin- 
gent Past 
Transfers 

Mean 

Mean 

Rank 

Response 

2.11 

2.79 

2.21 

2.59 

2.06 

3.24 

2.75 

3.00 

1.67 

2.11 

2.06 

3.38 

2.06 

3.11 

Establish  Tech- 
nology Transfer 
Agency 

Mean 

Mean 

Rank 
Respor  se 

3.03 

3.68 

3.18 

3.71 

2.62 

3.47 

3.00 

3.33 

3.33 

4,00 

3.13 

4.00 

2.17 

3.00 

n 

38 

17 

17 

6 

9 

8 

9 

Friedman  Test  Statistic 

25.6 

9.2 

16.1 

.4 

12.0 

5.9 

13.1 

Kendall’s  Coefficient 
of  Concordance  (W) 

.168 

.135 

.237 

.018 

.334 

.184 

.363 

Degrees  of  Freedom 
(vx;v2) 

4;  146 

4;62 

4j62 

4;18 

4;  30 

4;26 

4;  30 

F Statistic 

7.48 

2.49 

4.98 

.091 

4.02 

1.58 

4.56 

Level  of  Significance  (a) 
C1.7"5  = 1.7  x 10"5,  etc.) 

1.7**5 

- ,-2 
3.2 

1.5~3 

9.9-1 

-2 

1.0 

2.1"1 

5.4-3 
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TABLE  F-VIII  (CONTINUED) 


Question  19:  Please  indicate  the  extent  that  you  agree  or  disagree  that 

these  ideas  have  enough  merit  to  warrant  further  consider- 
ation, 


Respondents 

GS  15 
LC  & 
Above 

GS  14 
Maj  & 
Be]ow 

Over 

20 

Years 

Under 

20 

Years 

More 
Than 
2 Yrs 

Less 
Than 
2 Yrs 

Product  Divi- 
sions Rate  Labs 

Mean  Rank 
Mean  Response 

3.65 

4.08 

3.43 

4.57 

3.66 

4.63 

3.33 

4.07 

3.64 

4.67 

3.31 

3.92 

Labs  Rate  Pro- 
duct Divisions 

Mean  Rank 
Mean  Response 

3.50 

3.77 

3.02 

4.24 

3.47 

4.37 

2.87 

3.67 

3.21 

4.24 

3.19 

3.77 

Both  (1)  and  (2) 

Mean  Rank 
Mean  Response 

3.31 

3.77 

3.21 

4.38 

3.53 

4.47 

2.90 

3.73 

3.36 

4.38 

3.08 

3.77 

Make  Future 
Funding  Contin- 
gent Past 
Transfers 

Mean  Rank 
Mean  Response 

2.12 

2.85 

2.14 

2.95 

1.63 

2.79 

2.77 

3.07 

1.95 

2.95 

2.42 

2.85 

Establish  Tech- 
nology Transfer 
Agency 

Mean  Rank 
Mean  Response 

2.42 

3.00 

3.19 

3.95 

2.71 

3.63 

3.13 

3.53 

2.83 

3.62 

3.00 

3.54 

n 

13 

21 

19 

15 

21 

13 

Friedman  Test  Statistic 

12.8 

13.5 

29.5 

2.15 

21.12 

3.66 

Kendall's  Coefficient 
of  Concordance  (W) 

.247 

.161 

.388 

.036 

.251 

.070 

Degrees  of  Freedom 

<Vi;v2) 

4;46 

4;  78 

4;70 

4;  54 

4;78 

4;46 

F Statistic 

3.93 

3.84 

11.42 

,520 

6.72 

.907 

Level  of  Significance  (a) 
(7.9-3  = 7.9  x 10~3,  etc.) 

7.9"3 

6.7"3 

3.4**7 

7 ,2_1 

i.r* 

4.7-1 

L 

s» 

N 


ti 
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